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PREFACE 



The Course Development and Course Evaluation reports (TR 5,9 and 5.10) 
are the principal reports which describe the physics program delivered under 
Contract N00600-68C-0749 to the U.S. Office of Education. 

Three versions of the program were developed. The initial (1968) 
version received a tryout at the Academy during the last part of the Fall 
of 1968 and during the Spring of 1969. The second (1969) version had a 
tryout in the Fall of 1969 , and the final (1970) revised program was in- 
stalled in the Fall of 1970. 

The Fall 1968 materials were the first rough draft version and were 
used with only a few students in order to determine level of expectation, 
quality of materials and time requirements. The second tryout involved 
a considerably greater amount of material and lasted for the entire 
Fall 1969 semester with approximately one hundred students. 

On the basis of performance and student preference data the final 
version of the course was derived. Major changes in course content and 
operation were introduced in 1970. At this time. Academy faculty had the 
first opportunity to develop materials according to the methods detailed 
in the Course Development report. The faculty materials development 
effort was concerned principally with adding content to the original 
course in order to meet new Academy curriculum requirements which expanded 
the basic course to two semesters. 

The course as delivered is self-paced, independent study, multimedia, 
computer ot manually managed, introductory classical physics. It can be 
used at the U.S. Naval Academy with any number of midshipmen or at any 
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other institution having a need for the content contained in the objectives 
(as listed in TR 5.2.1 and 5.2.2). 

The Course Development report, TR 5,10, sets forth the procedures for 
developing new courses or adding content to existing ones. Capable pro- 
fessionals, through the use of this model, can design and develop self— 
optimizing courses or segments of courses . Evaluations of developmental 
processes are also detailed in this report, 

Summative evaluation is contained prinei ally in Technical Report 5,9, 
Course Evaluation, and to a lesser extent, in TR 5.0, Statistics; 1969 
Experiment. A study and evaluation of the classification of objectives 
is detailed in TR 5,8, Classification of Learning Objectives, 

\ PRINCIPAL REPORTS 

\ 

\ 

TR 5.10 \ Course Development A description and evaluation of the develop- 
ment procedures . Missteps and failures are documented as well as successes. 
TR^5,9 Course Evaluation An examination of improvement in the revised 
program, ant^ a comparison with conventional instruction, 

TR 5.8 Classificatio n of Learning Objectives Introduces and evaluates 

\ 

a classification hierarchy based on complexity of hard science objectives, 

\ 

\ 

AUXILIARY REPORTS 

TR 5 .0 — Statistics: 1969 Experiment A statistical analysis of data from 

the 1969 tryout , 

5. 1 Course Description An overview to put the course in perspective, 
(Included in TR 5,10.) 

TR 5,2,1 Co urse Objectives Each of the performance objectives is repre- 
sented by a problem so that the level, scope, and assessment measures are 
described in unambiguous form. 
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TR 5.2.2 Course Structure and Sequence The topical sequence of objectives 
including the defcision processes which led to this sequence. 

TR 5,3 Test Item Bank A compilation of criterion check items and diagnos- 

tic test items identified by terminal objectives. The item bank includes 
multiple questions for each terminal objective and item statistics collected 
during the tryout conducted in the Fall of 1969. 

.SL 5.4a Management System Repor t A description of course implementation 
procedures recommended by the contt actor and the method of presenting 
feedback. Record-keeping procedures and the forms on which records can 
be kept are described. 

TR 5.5 Revision Process Documentation A description of the specific 
empirical revision activities, rationale for these activities, and a 
compilation of the data upon which revision decisions were made. 

INTERIM REPORTS 

The interim reports submitted in 1969 describe the objective con- 
siderations and rationale for the Fall 1969 tryout. These reports include: 
TR 4 . 7 Rationale for Sequencing Objective s 
TR 4,7.1 Evaluation and Validation Design 
TR 4,7.2 The Validation Process 

TR 4.9 Design for Selection of Strategies and Media 
TR 4 . 3 Course Revision and Restructure 

TR 4,12 Weekly Course Segment Documentation, Weeks A through 0. 
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COURSE DESCRIPTION 






A systems approach was applied to the development of a multimedia 
computer managed course in college physics for the U.S. Naval Academy. 

Student Population 

The students are Academy midshipmen who are generally engineering 
and applied science majors. Their college experience includes one year 
of chemistry and introductory calculus. Clearly, this is a highly select 
and well motivated student population* The following is a brief descrip- 
tion of the course* 

Goals 

Several purposes are served by the program’s development and opera- 
tion* The Academy is provided with a cost-effective physics course 
designed with the most modern educational technology. The experience 
garnered in the construction of the program is incorporated into a 
course development model to serve as a prototype for construction of 
similar programs in other hard science courses. Extensive record 
keeping capabilities of the program allow it to serve as a vehicle for 
educational research* 

Characteristics 

The course is individualized, self-paced, and self-healing. By 
offering media optional routes through the learning materials, each 
student's learning experience is individualized, A student can use 
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any learning materials on an individual basis and , within practical limits 
at any time which is convenient for him. Likewise, within broad time con- 
straints, the student can progress through these materials at his own pace 

The self-healing aspect of the program pertains to learning materials 
and processes. These are improved by an iterative process of successive 
tryouts and revisions. In this way, the program undergoes an empirical 
optimization procedure. 

Content and Level 

The content and level of the course is generally described as intro- 
ductary college physics with calculus for sophomore students of science 
and engineering. 

Most standard topics in introductory classical physics are included 
in the two semester course: mechanics, wave phenomena, electricity, mag- 

netism, and optics. One unusual feature is the omission of heat and ther- 
modynamics in favor of more intensive developments in mechanics and optics 
to suit the Academy's particular needs. 

Systems Approach 

Optimization of the program must be attained empirically, since no 
satisfactory predictive theory of educational psychology is known. This 
situation Is well suited to a systems approach when the output of the 
system can be fed back to modify the system input. 

Behavioral Objectives 

This approach requires that the course objectives are clearly de- 
fined and measurable so that the output of the system is quantitative. 



Consequently * over a thousand measurable behavioral objectives (MBOs) 
were developed to. completely specify the performance which the program 
should elicit, MBOs fall into two categories: terminal objectives (TOs) 

which describe the desired final student behavior , and the subordinate 
enabling objectives (EOs) which are steps toward the terminal behavior. 
The TOs constitute a complete description of course content and are 
represented in the course by central aove problems. When a student can 
answer a core problem correctly, he is said to have achieved that TO, 

In this way a student knows exactly what is expected of him as to con- 
tent and level of proficiency. 

When a student cannot answer a core problem after a single exposure, 
he can execute subordinate enabling problems which correspond to the EOs. 
At the end of an enabling sequence, the student is presented with another 
version of the core problem to check has achievement of the TO, All these 
problems are contained in the Problem and Solution Book volumes. 

Media Components 

Videotape presentations are available for forty-nine topics. These 
tapes average about fifteen minutes apiece. Illustrated texts and talking 
books (taped voice-over illustrations in book form) are available with 
essentially the same information content as the videotapes. (Computer- 
assisted instruction (CAI) was initially included as a parallel path for 
topics in mechanics, primarily to be compared with the other options for 
cost effectiveness, and was deleted as a learning option in the final 
revision.) Conventional physics texts are also included among the avail- 
able learning materials. 

4 
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Tes ting 



Each student is provided with a s tudy guide which directs him through 
the problem books and various media assignments. When the student com- 
pletes a specified assignment (roughly approximated by a chapter in Halli- 
day and Resnick) , he schedules a progress cheek (test) on the material. 
This criterion check does not influence his grade but is used for manage— 
and remediation purposes. Grades are determined by quarterly tests 
and final examinations for each semester. 



Remedial s 

The most extensively used remedial material is that which provides 
immediate feedback to the student as he progresses through the learning 
materials. For every problem answered erroneously, a correct solution is 
immediately available. Minor remediation is accomplished by distributing 
a remedial sheets associated with each problem missed on the progress check. 
These sheets have’ a statement of the appropriate core problem together 
with references to pertinent auxiliary material. More serious remediation 
is provided by individual tutorial sessions with a professor. 

Laboratories 

The laboratories have as their objectives the measurement of funda- 
mental physical quantities, including the processing and recording of these 
data with an error analysis. An innovative aspect of the data analysis 

that a dialogue may be established between the student and the computer 
which would culminate in the student T s achievement of the objective* This 
computer dialogue laboratory format is not an essential element of the 
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multimedia learning materials, and in the current implementation the 
Navy Physics staff is using a conventional laboratory experience for the 
self- paced course , 
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THE COURSE DEVELOPMENT EFFORT 



The course development process was seen at the outset as the con- 
struction of a system which would provide feedback fox iterative revisions. 
Subject specialists began with essentially the same content as the tradi- 
tional introductory physics course; through a refinement process of 
testing* decision making* and alteration* course designers expected that 
a more efficacious product would evolve. The actual task was more exten- 
sive than this original conception. 

The first hint of a fundamental difficulty came during the original 
development of MBO lists. The writers intended to fractionate terminal 
objectives into their most elemental steps. The testing of every item* 
however* would far exceed any reasonable allocation of study time. 

Reduction to the smallest possible steps thus had to be rejected on 
purely pragmatic grounds “—there was simply too much material for that 
approach . 

A general philosophy was adopted for course development called the 
lean appvoaah^ whereby learning packages were constructed with the fewest 
possible embellishments which would meet the stated objectives. Data from 
subsequent tryouts would then identify the weak areas , and revision would 
supply the necessary additional instruction and refinement. Tills approach 
has several virtues; it is economical because superfluous material is not 
developed* it is an aid to revision analysis because trouble spots are 
most easily detected in a spare format, and it allows for maximal improve- 
ment in the most troublesome areas while it avoids excessive and tedious 
development in aieas of high performance. 
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Ihi', lean approach was Invoked to resolve the question of how small 
the enabling steps should be for a given terminal objective. Each terminal 
objective was therefore supported by a minimum number of enabling objectives 
the basic steps in reaching the terminal objectives. These enabling objec- 
tives mav require one or more minute steps for their own achievement. 
Terminal objectives singled out due to poor performance would have their 
enabling objectives more finely divided. 

Although 1968 final exam performance of the students in the physics 
program compared well with the students in the conventional course , cri- 
terion based performance was disappointing. Since the weakness seemed 
rather evenly distributed over most of the topic areas , it was reasoned 
that a uniform expansion of enabling objectives was the appropriate tonic. 
Knnb 1 i ng t oblems were increased from an average uf eleven to twenty-four 
per segment. Earlier qualms of course aevelopers were forgotten as the 
systems approach crank was turned and the conclusion was reached that 
"more is better" for all the required instruct' ona materials. 

Agnin in 1969 the performance of students in the physics program com- 
pared well with the conventional group on the final examination. But this 
time, in spite of the refinements and additional enabling problems and 
instructional packages, the program met with near disaster. Early in the 
semester a group of midshipmen complained that the course was excessively 
difficult , and requested to be removed from the program. Only a large 
scale shifting of problems to "enrichment 11 and optional status saved the 
project . 

The principal problem was clearly not one of refinement. Under the 
scrutiny of continual testing it became manifest that linear Instruction 
for all objectives was far too demanding of students' time. Indeed, the 
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corroliary followed that conventional physics courses are also too crowded 
for the a ot ted ■ credit and time— —the difficulty is simply not. perceived 
by limited monitoring. Thus evaluators arrived at the conclusion long 
known to most students: he hard sciences are different and more diffi- 

cult than other subjects. 

Perhaps part of the reason lies in the observation that the intro- 
ductory texts in hard sciences easily contain as much textual information 
as a standard textbook in world history — but the acquisition of all this 
factual knowledge is only a beginning. Hard sciences are problem oriented 
and demand the development of problem solving skills. An adolescent who 
re ligiously studies Hoi Rod magazine still must learn to drive! the stu- 
dent scientist may learn every physical fact in his textbook, but he will 
surely fail his examinations without having practiced problem solving. 

The excessive time required for completing the course work was the 
greatest single difficulty encountered in the development of the physics 
program. The trouble was not clearly appreciated until the 1970 revision. 
It was then obvious that the importance of this factor had been unantici- 
pated and was only minimally controlled and recorded. Course designers 
were in the embarrassing position of having prepared for a refinement 
process when a major reconstruction was required. 

In addition to toe factor of student time in study, two other 
quantities appear to have this important maorovariab le status: student 

motivation and course level. Motivation is characteristically high at 
the Naval Academy , and thus did not present any special problem for the 
physics project. The program evaluators suspect that motivation would be 
a greater problem at the compulsory education level. Nevertheless, they 
believe that putting the course on a self-paced basis is conducive to 
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student motivation because there are rewards intrinsic in such a system; 
the students may advance at their own rates with tha satisfaction of 
knowing how well they are succeeding as they progress. 

The notion of course level is subject to considerable ambiguity in 
the hard sciences a ^ stated in common curriculum form, A curriculum for 
a high school physics course might be indis tinguishabie from that of an 
introductory college course as judged by chapter and section headings# 

The process of labeling subject matter is insufficient to describe the 
intended depth of the course. 

This problem has been recognized for some time, and has been 
approached by stating objectives behavi orally , usually in verbal form. 
Since the hard sciences are problem solving oriented, we specified 
level in problem form; that Is, all objectives were defined by sample 
problems. In this way, the student knows immediately what behavior is 
expected of him on a criterion test. 

A distinction has been drawn between macrovariables and mi qipq v ar*i — 
ablest which pertain to the small adjustments of content, presentation, 
and sequencing. In the hard sciences these microvariables are far more 
precisely defined than in other disciplines . 

In order to relieve the demand on students’ study time, the possi- 
bility of reducing course content was considered. One of the contractual 
requirements, however, was to develop materials corresponding to the 
conventional S211 course. This meant that clL Z students were required to 
study the same topics and take the same final exam. The program developers 
were responsible, then, for maintaining all the topics in the curriculum, 
although in conventional courses instructors are at liberty to adjust topic 
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coverage through omission and abridgment. To maintain the feedback 
requirement^for course revision, it was required that students pass through 
all the material. 

To mitigate the problem of excessive demands on students s a new format 
was constructed * The first objective to be met in any topic is a oox*& 
problem— a terminal objective in problem form, A completely detailed 
solution follows immediately , These provide the student with the clearest 
statement of the performance required and provide an excellent advance 
organizer , This format allows a direct assault on the terminal objective 
so that detailed enabling objectives may not be necessary. Another core 
problem, closely parallel to the original, is presented at the end of an 
enabling sequence. The forward branching economises the student ¥ s time 
and avoids his laboring unnecessarily over more problems. 

Our solution, then, is to provide the student with a terminal objec^ 
tive and to show him the correct way to achieve it, Th^ student thereby 
focuses on the essentials he needs to meet the ob j e< f ve . If this is 
sufficient , he need not expend valuable time and effo '■•n this objective, 
but may proceed immediately to the next core problem . 
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COURSE OBJECTIVES 
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SELECTING COURSE CONTENT 



The first step in course development is to prepare a framework of 
general topic headings which loosely define the course content. This 
Is easily accomplished in the hard sciences by selecting chapter and 
section headings from some standard textbook. The edition of the text** 
book should be at least two years old to insure that the authors have 
had a chance to correct the most obvious deficiencies of earlier 
printings * 

Instructors who will administer the course should be included in 
the selection procedures whenever possible. Our experience has been 
that instructors exhibit considerably more drive and interest in the 
program when they have participated in the development phase. The 
chapter and section headings should be arrived at by a consensus or 
some compromise. Autocratic rulings by a technical director or depart- 
ment chairman, however wise or correct, should be avoided. It is likely 
that in order to satisfy everyone, more topical material will be included 
than might be chosen by any one instructor. 

The committee of instructors must bear in mind the preparedness of 
the incoming students and the role of the program as prerequisite to 
other studies. Naturally, any constraints on the subject matter must 
be considered at this time. The subject matter areas In the physics 
program, for example, were dictated by specific needs of the Naval Academy 
and include most fundamental topics In basic mechanics, and basic electri- 
city and magnetism. 
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Stated broadly, the mechanics topics are: kinematics, vector 

algebra, Newton f s laws of motion, gravitation, work, conservation of 
energy, impulse, conservation of linear momentum, and collisions* The 
electricity and magnetism topics include, charge and Coulomb's law, 
the electric field. Gauss's law, electric potential, capacitors and 
dielectrics, resistance, currents. Ohm's law, Kirchhoff's rules, magnetic 
field. Ampere's law, Faraday's law, Lenz's law, law of Biot-Savart, and 
inductance . 

Textbook publishers usually identify course material as "hard" or 
"soft" ; hard refers to those for which a well defined course content has 
evolved* Soft subjects are more amorphous, and one text may differ from 
another even though they are directed to the same course and student 
population. The physical sciences clearly fall into the hard category. 

Any significant variations from the highly structured course content 
is likely to meet with considerable opposition from some professors, 
especially when they did not participate in making the change. An unortho- 
dox development in symmetry, for example, had to be deleted after the first 
tryout due to the course instructors' dissatisfaction. Course developers, 
however, contended that students would find it interesting and that it 
would greatly simplify the approach to conservation of energy and momentum. 
We suggest that well established topic headings in standard texts be taken 
as the base from which to work. Departures should be promoted by their 
adherents to the other instructors before course material is developed and 
forced upon unrecep tive faculty. 

Our remarks about the well established topical content of hard 
science courses have to be tempered somewhat for less evolved sciences 
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such as molecular biology. In such cases the broad course content out- 
line may have to be established with little or no reliance on a standard 
text, and compromise will have to play a larger role. 



Discriminating between "subtopics" and "major topics" must be left 
to the course developers. We had no difficulty in making such assign- 
ments, and we assume that it is equally simple in the other sciences. 



Most of the major topics correspond to chapter headings and subtopics 
correspond to section headings in a conventional textbook. For example, 
under the following listing: 



COLLISIONS 



Definition of Impulse 
Impulse and Momentum 
Collisions in One Dimension 
Collisions in Two Dimensions 



} Major topic (chapter heading) 



>■ Subtopics (section headings) 



The topic headings are hardly likely to be understood by a non- 
scientist. This is unimportant since course content listings ere used 
for information only by course writers and instructors. These specialists 



generally have an excellent idea, due to precedent and experience, of 
the content implied by such cryptic topic labels . 

When these lists are written and agreed upon, they must be repro- 
duced in sufficient quantity to serve as working check lists for all 
the course developers. 
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WRITING COURSE OBJECTIVES 



The need for stating objectives has been emphasized over the past 
years and now enjoys widespread acceptance* Attempts at specifying 
performance have resulted in different classification schemes, all with 
the intention of clarifying for instructors, course developers, and 
students what will be accepted as an indication that a student has 
learned. Because we cannot see learning directly, we must base our anal- 
ysis of learning on more concrete evidence; we take as our evidence a 
modification of behavior, a change that can be demonstrated and measured* 

After determining the general course outline for the physics program, 
we began writing Measurable Behavioral Objectives (MB Oh) * To assist the 
course writers in listing MBOs that were behavioral, we prepared sample 
items and lists of action verbs* Unfortunately, we later realized that 
using words such as define * convert* perform* apply* select* state* 
indicate* write* does not insure specific measurable behavior. 

For example, some of our earliest efforts produced these objectives; 

Define a derived quantity as used in physics* 

Convert length in one metric unit to length in any other metric 
unit * 

Describe qualitatively the operation of a quartz crystal clock 
and give the order of magnitude of its precision. 

Indicate understanding of the procedure used to adjust local 
clocks by means of time signals from WWV . 

Write any given number in scientific notation. 

Apply area and volmne formulae to the solution of problems. 
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Clearly they require action, but are not precise, do not specify the 
conditions under which competence must be demonstrated, and do not indi- 
cate degree of competency to be achieved. 

We grouped the 3000 MBOs we prepared into Terminal (TO) and Enabling 
(EO) objectives, and defined TOs as the desired final behaviors. The 
EOs , the more specific smaller blocks leading toward the TOs, represent 
the transitional type of skill or knowledge which is believed to be a 
precondition for success on the TO. They are not all necessarily of 
equal importance or of equal difficulty to attain, and they may or may 
not build upon each other; it is sufficient only that they facilitate 
the mastery of their associated TOs. 

This procedure of identifying objectives as terminal, or enabling, 
or interim, etc., is often time consuming and confusing, since TOs them- 
selves can be EOs to still more complex TOs. Grouping them in this way, 
however, did provide us with a method of reviewing our lists and organ- 
izing the material efficiently . It revealed those EOs which were irrele— 
^®nt to the TO and those which were redundant; for example, "Convert 
units of ft/min to ft/sec and "Convert units of meter s/min to meters/ 
sec" do not require different behavior for their execution. In addition, 
by ordering objectives in this way, we could more easily spot omissions 
and prune those objectives which were of purely cultural orientation, 
allowing for more concentration on difficult or more important objectives. 

The revised lists, reflecting a consensus reached through joint ses- 
sions with the Academy and both centers of NYIT , directed the modifica- 
tion effort of our course materials. 

We found, through running our course, that requirements for manage- 
ment and revision were best satisfied by casting the objectives into the 
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form of a representative test question. Our verbal statements of objec- 
tives 3 and their lack of precision ? had resulted in a lack of clarity. 
Students were not able to assess the objective's leva! and scope, and 
so were not certain exactly what was expected of them. Professors with 
little specific classroom experience, regardless of their subject matter 
expertise, could not be sure to what extent any content was to be covered 
Although there was professorial consensus on a list of objectives, there 
was little agreement on whether the objectives were achieved: evaluation 

of student progress was less than standardized. 

Use of verbal objectives revealed another more serious, though less 
obvious, hazard. Enabling objectives are clearly discerned by the .course 
instructor or designer on ' rV >en he executes the behavior called for in 
the Terminal Objective. Each necessary step toward terminal behavior is 
then, by definition, an Enabling Objective. When a Terminal Objective 
is "fuzzy" the requisite steps for its achievement are even less clear. 
Our experience has been that this situation leads to poor ordering of 
Enabling Objectives under a Terminal Objective, a number of redundant 
Enabling Objectives, and a few omissions of Enabling Objectives. 

We reformulated the objectives into problem form to eliminate these 
difficulties. With a precise problem to represent a Terminal Objective, 
any subject matter expert can perform the steps for its solution and 
identify the individual steps as Enabling Objectives, Moreover, speci- 
fying objectives in problem form indicates to students what behavior will 
be considered acceptable and under what constraints their behavior must 
be evidenced , 



For the specifies of problem construction, see the section on "Test 



Items . 
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Core Problem Development 



Core px'oblems are generated from the course content list. All the 
subtopics under a major topic heading should be undertaken by the same 
writer (or pair of writers) so that connectedness and uniformity can be 
assured. Otherwise there is a danger of overlap or omission of essential 
material between adjacent subtopics. Furthermore, a single writer can 
often Introduce a connecting theme throughout a major topic | it may 
take the form of a familiar physical situation recurring with embellish— 
ments j or a sequence of problems addressing entirely dissimilar situa- 
tions but having the same analytical solution. 

In developing core problems, the writer should be guided by the 
following rule: Write the test problem which is the best measure of 

a student f s achievement in the given subtopic . By "best measure" we 
mean the appropriate scope and level of the test question as well as 
content. The ideal being sought here is the perfect examination 
question; one which, when successfully solved, indicates that the 
student has mastered the subtopic within the course structure. 

It is to be expected that more than one problem will often be 
needed to cover a given subtopic, A good rule— of —thumb , however, is 
when more than one set of problems satisfy your requirements equally 
well, choose the set which places the least burden on the student. When 
one of the objectives is a synthesis of different (new) elements, we 
found it best to write core problems for each of the elements and for 
their synthesis as well, 

A good deal of decision making is involved in arriving at a set of 
core problems. o. an example, consider writing core problems for the 
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subtopic Collisions in One Dimension . Several possibilities immediately 
occur to someone who wants to exhaustively test this topic. Collisions 
can be elastic (energy conserving) or inelastic, and various degrees of 
inelasticity can exist. In the most general one-dimensional collision 
between two bodies, both objects are moving (in the laboratory frame) 
when they suffer an inelastic collision. The final products of the 
collision arc moving fragments of various masses, A collision of such 
generality is not addressed in this course. 

The actual core problems which were believed appropriate to the 
course pertain to totally inelastic or totally elastic collisions where 
one of the objects is originally at rest: 



1, A railroad car of mass 1000 kg is rolling down a track at 
3 m/sec. It strikes a stationary car of mass 2000 kg. If 
the two cars couple together, what is the speed of the combi* 
nation immediately after the collision? 



2, In a one-dimensional elastic collision between two objects, 

mass m 2 is initially at rest. If Uj = 1 km/sec, and m 1 — 2m 2 , 
what is the final velocity of m^ ? 



3. 



A 1/ 2 -ounce bullet traveling horizontally with a 
of 1500 ft /sec strikes a 5— pound block suspended 
point with 5*foot, massless, inextensible cord* 
remains embedded in the block, how high does the 
(Neglect air resistance.) 



muzzle speed 
from a fixed 
If the bullet 
block rise? 
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Problems 1 and 3 overlap in that they both treat collisions with coupling. 
However, problem 3 forces a discrimination to be made as to the circum- 
stances under which mechanical energy and momentum are conserved* The 
physics of problem 1 is entirely contained within problem 3 — t he "new" 
objective contained in 3 is a synthesis of two concepts. 

Course writers will usually have considerable experience in writing 
test questions; some elements of constructing problems are presented in 
the Test Items section. When a writer has completed a set of core prob- 
lems, he should submit them to review by the same people who decided on 
course content. It should be expected that some problems will be added 
and some omitted. 
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CLASSIFYING LEARNING OBJECTIVES 



Writing learning objectives for highly quantitative hard science 
courses requires careful specification of an elusive measure of level* 
For example 5 high school and college physics courses may cover essen- 
tially the same topical material, but with a profound difference in 
requirements and expected performance* 

In order to categorize level in a measurable way, we attempted to 
develop a elassif ication of learning objectives based on a nonarbitrary 
measure of complexity. Basically our measure of complexity is the nuui- 
ber of algebraic equations required for the solution of the problem*' 

This measure was found to be a good organizer and predictor* of student 
performance , and has important advantages over a classification based 
on student difficulty. 

Complexity is a nonarbitrary objective standard. It is a quality 
inherent in the task* Depending upon one’s qualifications, it may be 
easy or not to achieve* In either case, complexity level can be deter- 
mined in the absence of performance measures and does not require 
knowledge of preceding learning experience. 

Difficulty, on the other hand, is both a more relative and more 
subjective measure. It is often determined by professorial experience 
with prior student performance on similar items, or by analysis of 
student performance, often long after it would be useful for formative 
evaluation , 

The use of complexity has been found to be a useful predictor of 
problem difficulty (see TR 5.8). Since a measurable objective must 
specify performance, and performance is highly dependent upon complexity , 
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1 follows that complexity should be specified for hard science objec- 
tives Even if objectives are stated in problem form, the complexity 
category of the problem should be known in order that variations of the 
problem (for use on a final exam, or for comparison purposes) can have 
equivalent complexity - 

The classification by complexity is a useful systematizing agent. 

As a result of our investigations, we concluded that enabling objectives 
should not be more complex than the related terminal objective* EOs 
should facilitate success on the terminal objective; as the complexity 
of an EO increases, it is likely to produce a decrease in performance. 

Our original classification system (see attached sample) , based on 
Bloom’s taxonomy, was found to be inappropriate for one or more reasons; 
too general, covering more kinds of behavior than were apparent or 
necessary to test in a physics problem-solving course; too ambiguously 
or nonob jeetively defined, requiring suppositions about internal thought 
processes or arbitrary assignment to subclassifications; too history 
dependent, requiring previous learning experiences as a base. For these 
reasons it was decided to adopt an objective measure, problem complexity, 
for categorization purposes. 



For the physics project, three categories of objectives were identi- 
fied, and a catch-all gray category was invented* They are listed in 
order of increasing complexity. 

(1) Zero Step Problems; Those questions which do not require 

any mathematical manipulation* Recall of a fact or defini- 
tion, or the recognition of an object, fact, or defini- 
tion fall into this category. Even difficult conceptual 
problems or associations are categorized as zero step ques- 
tions whenever there are no algebraic steps involved. All 
O 
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ORIGINAL GLASSIFICATION SYSTEM 



29 QUESTION CATEGORIES 

29.1 Purpose - Hie structure of question categories 
has been erected to provide another meaningful 
source of input for computer data analysis and 
management* ^ The plan calls for the assignment 
°f all questions used in the course (diagnostic, 
criterion, and administrative) to specific ed- 
ucational categories as presented below* It 
is anticipated that this system will, by anal- 
ysis of individual student perfor ma nce , yield 
information which can pinpoint weaknesses in 
the student* s thought processes, and thus pro- 
vide direction for the remedial work* 



29*2 Nature of the Category Structure — Study of 
the list of categories indicates that they 
compose a hierarchy of performance levels in 
which the capabilities at the higher levels 
rest upon those at the lower levels. Hence, 
the assignment of a question to a category 
will be based upon the highest level Into 
which it can be placed* 

29*3 List of Categories (From lowest to highest level) 



Level 1 : RR 

BN 

Level 2 : AN 

Level 3 s CN 

Level 4* SY 

Level 5: MR 



Recall and Recognition 

Facts , theories, laws and principles* 

Basic Mathematical Skills 

Basic operations! one-step translations 



Analysis 

Identification of elements 5 identifi- 
cation of relationships among elements § 
reordering and rearrangement of elem- 
ents for required operations. 

Comprehension 

Recasting of elements to meet specific 
needs} translation of elements into 
different terms as required} interpret- 
ation of relationships among elements* 

Synthesis 

Implications, consequences, corollar- 
ies} solution of Verbal problems} 
explanations of events and phenomena} 
derivations} formulation of Hypotheses} 
prediction. 

Mathematical Reasoning 

Links between synthesized groups} 
sequencing groups for assembly into 
a comprehensive whole* multi step- 
problem-solving. 
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word problems; that is , problems which are rxonnumerical 
and nonsymbol ic , are zero step problems. 

(2) One Step Problems: Those problems which require the 

solution of one algebraic equation for one unknown 5 or 

a single calculus operation (differential or integration) . 

(3) Multiple Step Problems: All problems which require the 

solution of more than one algebraic equations or more 
than one calculus operation* 

Some examples follow: 

i 

Zero Step 

At time t & 0 , a puck is observed to move on a frictionless hori-* 
zontal table with a speed of 40 ft/sec. After two seconds the speed 
of the puck is : 

One Step 

A constant force of magnitude 100 lb is required to move a block 
along a horizontal floor with constant speed of 4 ft/sec. The force 
is directed along the motion of the block. Calculate the power deliv- 
ered by this force. 

Multiple Step 

A section of level roadway has a radius of curvature of 100 m and 
is expected to handle traffic at 10 m/sec. What minimum coefficient of 
friction prevents skids at this speed? 

For more examples % see Appendices A* B, and C in TR 5.8* 
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We do not distinguish between two, three, or more step problems, 
since we can analyze the number of steps only in the intended behavior, 
not the actual behavior. By eliminating the distinction between, say, 
two and three step problems, we minimize this difference between intended 
and actual behavior. 

In addition, when more than a single operation (step) is required 
to solve a problem, even experts frequently disagree as to the "best" 
way to solve it and the number of steps required. Clearly, ambiguities 
in the step counting process are much more likely to occur in multiple 
step problems, 

A fourth catch-all category included all problems which were judged 
not valid or which did not fall precisely into one of the three categories 
Problems which had errors, insufficient information, or ambiguities were 
relegated to this category , as were many graphical problems which require 
geometrical steps rather than algebraic steps. A majority of problems 
lacking clear definition required simple arithmetic operations; we did 
not want to equate trivial arithmetic computations with algebraic steps. 

Of course, all the valid problemr can still 1 a assigned to categories 
by making the above definitions more exhaustive and detailed, 

A much more detailed description and discussion of the classification 
hierarchy is contained in TR 5,8, 
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SEQUENCING SUBJECT MATTER 



The sequencing for a majority of topics in hard science are dictated 
by the fact that the concepts or techniques contained in one topic are 
often prerequisite to the treatment of a second topic* Terminal objec- 
tives of an earlier topic are prerequisite to succeeding topics* We 
will refer to those subject matter sequences, which are mandated by this 
condition as "subject matter constrained sequences" (SMCS). Broad SMCS 
for the physics program are shown in the accompanying diagram* The topics 
represented 3 recommended for inclusion by the Academy staff, are those 
included in the conventional S211 course outline* 

The remaining ordering of the material (within the constraints 
imposed by the subject matter) is established according to the criteria 
of "inclusion’' and student difficulty. The inclusion criterion requires 
that when Topic B has most of the terminal objectives of Topic A as 
enabling objectives, then Topic B must immediately follow Topic A* This 
criterion receives the highest priority in subject matter sequencing, 
because evidence indicates that forgetting is a function of intervening 
learning rather than a function of elapsed time and immediate use of 
prior terminal objectives tends to reinforce them. 

The following are sequences established by the inclusion criterion 
in the physics course: 

Kinematics -► Relative motion 

Work and energy Potential energy and conservation of energy 
Conservation of momentum =*■ Collisions 
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Circular motion (kinematics) Circ ilar motion (dynamics) 

Charge and Coulomb 1 s law ^ Electric field Gauss 1 s law 
Currents and Ohm's law -*■ Kirchhoff 1 s rules 

Student difficulty is employed as a sequencing criterion so that 
those topics which* in the experience of the teaching physicists, are 
most difficult for students are placed nearest the end of the course. 
This rule is based upon the assumption that a student gains maturity 
(facility in mathematical manipulations and rapid recognition of the 
principles and techniques involved) and sophistication (a backlog of 
concepts to rely upon) as the course progresses. Since new material 
should be less difficult for the more mature and sophisticated physics 
student, the difficult topics should be put toward the end of a sequence 
The student difficulty criterion is sufficient to dictate the 
remaining topic sequencing to be contained in the Problem and Solution 
Books , 
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INFORMATION PANELS 



Information Panels usually provide an overview for the student, or 
supplement information offered elsewhere* They direct attention to the 
theory required for the core problem solution. Information Panels 
always describe the core objectives in a form which is comprehensible to 
the student (see attached sample) * 

Information panels also serve to present supplementary topics chosen 
on several bases, "Trouble topics , 11 those concepts which students inevi- 
tably find difficult to attain, can be identified by experienced instruc- 
tors and examined more fully here than in a usual presentation. In addi- 
tion, Information Panels provide an excellent opportunity for the profes- 
sor to detail a different approach to a conventional topic, or relate 
relevant experience. In early versions of the physics program, these 
functions were served by "Supplementary Notes 11 distributed throughout 
the problem books. 

The Supplementary Notes had the character of informal discussions 
about recondite points, often clarifying details and showing explicit 
examples, (A sample of Supplementary Notes is attached,) These were 
well received by students, so we attempted to maintain the same informal 
style in the Information Panels. 

The need for presenting learning objectives in a simplified form 
which is clear and comprehensible to students was recognized in the 
earlier versions. 

The students were given statements of the objectives at the begin- 
ning of each week of the course* These objectives were the same des- 
criptions that were initially developed by the course designers for 
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SAMPLE INFORMATION PANEL 

SEGMENT 7 



INFO KMATIQN P AN EL W ork Done by a Constant Force 



OBJECTIVE 

To calculate the work done by a constant force, that is, a force which 
varies neither in magnitude nor direction. 



In the simplest situation, where the force applied to a body is constant 
in both direction and magnitude and where the resulting motion occurs in 
a straight line, we define work as the product of the magnitude of the 
force and the displacement of the particle on which the force acts* 

Since force an^ 4 displacement are both vectors, care must be taken to. 
use a consistent system of symbols. In our work we will continue to use 
i for the position vector. Displacement will be designated by i so 
that a particle moving from position "r ^ to will undergo a displace- 
ment of Ar s s - ^ ~ - Thus, with this convention, dlt 5 dl? and the 

two differentials may be used interchangeably although ds will be the 
preferred form. 

The work W done by a constant force ^ acting on a body which moves 
through a displacement t is W = = Fs cos 0 in which 0 is the angln 

between the two vectors* 

If we designate the component of the force in the s-direction as , 
then 

F a = F cos 6 

and so W = F s s 

In working through the problems dealing with the work done by a constant 
force, you will be expected to 

(a) justify the conclusion that the work done by centripetal 
force on a particle moving uniformly in a circle is zero; 

(b) calculate the work done on a given mass when moved up an 
incline by a given distance; 

(c) find the work done on a given mass when lifted vertically 
over a given distance. 
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SAMPLE SUPPLEMENTARY NOTES 



(Vol. C) 



Supplementary Notes 



In the problems we have dealt with in this segment thus far p the 
"work done" was given or found to be a positive quantity. However , 
although work is a scalar quantity, it is convenient to assign 
a sign to it. This sign indicates which body does the work. If s 
for example, we are trying to find the work done by body A on 
body B and it happens that body B is actually doing work on body A, 
then the work will be a negative quantity. By convention 

the work done by a physical ‘’system" on its environment is taken 
to be positive. If the work comes out negative, then we know it 
is the environment that does work on the system under consideration 
In the following question the "system" will consist of a spring 
resting on a smooth table. One end of the spring will be fixed. 

The convenience of the convention on the sign of the work done will 
become apparent from this question. 



A spring is said to obey Hooke's law if the force necessary to 
stretch (or compress) the spring is directly proportional to 
the degree of stretching (compression). Within limits, most 
springs obey Hooke’s law. 




If the equilibrium position 
of the free end of a spring 
is x Q , the force ? f necessary 
to stretch the spring to a new 
position x is given by ' = 
k (x-x Q ) , where k is the force 
constant of the spring. Stick- 
ing to our convention, we 



talk of the force the spring 
applies to its environment. 
From Newton’s third law this 
force is ? = -? = -k(x Q -x) , 

or, written in scalar form, 

F - -k(x-x o ). 



communication with subject matter experts , Academy faculty, and other 
project staff , The objectives were not designed specifically to communi- 
cate the intent of the instruction to the student* When objectives are 
frequently incomprehensible to the student until after he has achieved 
them, students tend to receive far less information from the objectives 
than is desirable. The purpose of giving a student the objectives is to 
provide him with a concrete, identifiable goal, and to give him some 
basis for self-evaluation of progress toward that goal in terms of his 
own performance capability. If the objectives do not serve that purpose, 
a central component of the system is missing; if the students recognize 
the deficiency, they tend to not bother to read the objectives . A major 
rewriting of objectives was called for to produce a set of written des- 
criptions that would be comprehensible to the students prior to their 
contact with the study materials, and usable by the students as a pro- 
gress evaluation guide - 

The usefulness of the early student oriented objectives was obviated, 
then, by several faults: 

a) The statements of objectives were listed on a separate page 
(see the sample of Phase II descriptive objectives) * In order 
to le most effective, these objectives should appear just before 
the student encounters the associated learning material. 

b) The terms used in the descriptions were not previously de- 
fined. Thus, "work," "power , 11 "kinetic energy," and "work-energy 
theorem" were not yet defined at the time a student encountered the 
instructional objectives . 

c) The objectives were not stated in precise behavioral terms. 

It is clear that a student could not really tell wh^n he was 
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SAMPLE PHASE II DESCRIPTIVE OBJECTIVES 



( Vol , C) 



P E fi P F N T S F V F N 



ASSIGNMENTS 



Reading 



Homework 



HR Chapter 7* HR 7-8 

SW 6-1, 7 

SZ 7-1, 2, 3, 9, 10 

AB 6-1 , 2 , 3 , 5, 6 5 7; 27-1, 2, 3 



INSTRUCTIONAL OBJECTIVES 

Upon completion of this segment you should be able to 

(1) answer fundamental questions and solve problems 
pertaining to the work done by a constant, or 
displacement-dependent force (graphically and 
analytically, 

(2) solve problems involving the power delivered by 
mechanical system, and 

(3) solve problems involing the kinetic energy of a 
and the use of the work-energy theorem. 



Please turn to page 5, 




Indicates reading of prime importance, 
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"...able to answer fundamental questions and solve problems 
pertaining to the work done by a const ’"t, or displacement- 
dependent force." Compare this with the more behavioral 
statements which succeed it in an Information Panel: "In 

working through the problems dealing with the work done by 
a constant force, you will be expected to 

1. justify the conclusion that the work done by 
centripetal force on a particle moving uniformly 
in a circle is zero; 

2 . calculate the work done on a given mass when 
moved up an incline bv a given distance; 

3. find the work done on a given mass when lifted 
vertically over a given distance. 



O 
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PROBLEM AND SOLUTION BOOKS 



The Problem Solution Books, in their present form, contain 

Information Panels, core and enabling problems, and detailed solutions 
for all problems. We will discuss the precursors of the Problem and 
Solution Books and indicate the major steps in evolving to the current 
format. Hopefully, this treatment will help others avoid our earlier 
miss taps , 

Our first version of the F&S Book was a branched program which in- 
cluded all instructions to manage students through the course materials, 
Reading assignments, supplementary notes, directions to audiovisual 
materials, problems, and remedial frames were all contained in the 
original sixteen volumes. Three features were to undergo considerable 
change , the remedial function, the problem format, and the management 
function , 

Remediation had a central role in the early FSrS Books (see also the 
section on Remediation) * The problem statements were all in multiple 
step i ormat ; when a student selected the correct answer on an auxiliary 
answering device, a check (/) was displayed indicating that he should 
proceed to the next problem (see sample page of original problem book) , 
When an incorrect dis tractor was selected, a page number was displayed 
indicating the location of remedial inf ormation . On occasion, the 
distractor was sufficiently inappropriate to warrant the display of a 
cross (x) indicating M try again, " 

This remedial emphasis arose from an assumption that a substantial 
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Sample. Page of Original Problem Book 
( 22 * 1 ) 1 - 1 - 1. 0 

(22.2) Re a ding : HR l-l, 1-2. 

(22.3) Supplementary notes ; The story of the growth 

of measuring units Is a. fascinating one. From It, 

one gets an Insight into the ways that man hhs devised 
to overcome the problems he fa^ed when he began to 
acquire property, indulge in barter, and exchange 
services. Audio-visual number 1 (AV 1) is concerned 
with this narrative. 



(22.4) 

(22.5) 

( 22 . 6 ) 
( 22 . 7 ) 



(22,3) 



(22,9) 



( 22 , 10 ) 



AV: 1, The Growth of Measuring Units. 

AV Data? None 



QUESTIONS AND PROBLEMS 

1. This question is presented in AV 1. 

X A Unavailability of the measuring 
device, 

/ B Variability of the standard of 
measurement • 

X C Inconvenience of the selected unit, 

X D Absence of multiples and submult i 
pies of the chosen units, 

2. Why is length considered a fundamental quantity 

/ A Its defining operations are not 
based on physical quantities, 

1.1 B It is easy to measure 

1.2 C Its standard unit may be subdivided * 

into smaller standard units. 



3* Which one of the following is a derived rjiantitj 



1.3 A Time, 

1.4 B Length, 

1.5 C Area 



4, 'Which one of the following measures the same 
fundamental quantity as the cubit? 



X A acre 
X B gallon 
2.0 C rod 
1,6 D minute 
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fraction of students would get most problems correct immediately after 
having been exposed to the associated learning materials. The assumption 
proved incorrect for most higher level questions. Moreover, we imagined 
that the most probable errors could all be identified and addressed in 
the distractors and their associated remedials. Again, students were 
much more imaginative in committing errors than the subject matter 
experts were in anticipating them. 

Recognition of these defects was not immediate. The overall im- 
pression received from the data was simply that performance in the 
Problem Books was far short of expectations. Our use of the systems 
approach at this point was more mechanistic than thoughtful; the pl&n 
called for a more "fine grained" treatment of valid items with low 
performance. Consequently, the number of problems was more than 
doubled, each remedial was expanded to include considerable detail, 
and a full scale solution was provided for every correct answer solution. 

The burden that this expanded version placed on students' study 
time has already been mentioned in the Introduction. Achievement 
continued to be disappointing and a hard look at our data and pre^ * • 3 

led to revisions in the remedial functions and format of the Problem 
Book. 

Specific remedial frames were no more effective than one general 
remedial frame which told all students how to approach the problem in 
the correct way. We therefore eliminated remediation for specific 
incorrect choices and used an expanded discussion of the correct answer 
as both reinforcement and remediation for right and wrong answers, re- 
spectively. 
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This alteration eliminated a great deal of page turning, which 
the students and faculty all found objectionable, conserved student 
time in extraneous remediation, and eliminated the need for course 
writers to ’’second guess” the possible student errors. It also per- 
mitted the use of other than multiple-choice questions because incorrect 
alternatives were no longer a necessary part of the system. 

In the last revision it was recognised that objectives are best 
represented by problems. We termed the problems associated with TO f s 
core problems and each co~e problem had associated with it enabling 
problems corresponding to the enabling objectives, 

i o relieve some of the demands upon students, we constructed a 
new format, A core problem is presented first, before any enabling 
problems, and is followed by a detailed solution.* This provides the 
student with an exact statement of the behavior which is required, and 
is an advance organizer. If a student succeeds in solving the core 
problem, he may proceed directly to the next core problem* 

Enabling problems and their solutions* follow for those who were 
unable to treat the core problem. Finally, at the end of the enabling 
sequence, n variation of the core problem is presented, the core prime 
problem. The format for each objective, then, is: 

Core Problem (and Solution) 

Enabling Problem 1 (and Solution) 

Further Enabling Problems (and Sol „i ons) 

Core Prime Problem 



*These are nut located next in sequence in the Problem and Solution 
Book, however, but are on some scrambled page to prevent inadvertent 
disclosure , 
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This forward branching economizes the student ! s time and avoids his 
laboring over required sequences wh ‘ :h he has already mastered. 

The last major change from the original to the present Problem and 
Solution Book is the removal of all the management and reference 
functions. These functions are now relegated to a separate sheet, the 
Study Guide (see section on Study Guide), because it was wasteful to 
reprint the large F&3 ; ^ks for the purpose of making simple omissions, 
sequence changes, changes in references, or any similar alterations. 
Thus, the P&3 Books are now simply repositories of the more enduring 
Information Panels, problems, and solutions* 

H ow to Develop the P&S Books 

Ue assume that the core problem statements are available at this 
stage- The major steps in PckS Book construction, then, are (i) develops 
went of core problem solutions, (ii) writing enabling problems and 
solutions, (ill) development of core prime problems, and (iv) assembly 
and scrambling of material for the book’s construction* 

(i) Develonlng Solutions - — In developing a solution for a given 
core problem, ore should bear in mind that he is developing teaching 
material. As ruch, the solutions should emulate good tutorial practice 
of "walking the student through” the method of arriving at the answer, 
discussing each step in the process. This permits the student to attend 
to the specific information that he needs in the context of a problem 
solution * 

A recurring question is "how detailed and comprehensive should the 
solution be?" Again, tutorial practice is a good guide. Those particu- 
lars which have only recently been addressed in great detail may be 
treated rather briefly in subsequent solutions. The main emphasis 



should he directed to the newest elements of the objective. When a 
doubt exists, it is better to include more detail than may be necessary 
rather than less. 

A good problem writer requires a knowledge of prio, problems and 
solutions on the same topic. Clearly, a particular course writer (or pair 
of writers) should develop all solutions under a particular topic heading 
(usually corresponding tc a ch ter in a text) , 

The accompanying sample problem solution exemplifies some desirable 
features. The correct answer is provide first, the solution uses diagrams 
whenever they are likely to be helpful, and symbols are defined and used 
in lieu of specific numbers. The sample solution is brief in its treat- 
ment of the equations of motion, and does not show ay steps in the simul- 
taneous equations 1 solution. When the student sees this solution, he has 
received intensive exercise in one-body equations of motion, and the 
ability to solve pairs of linear equations is a prerequisite to the course. 
The new feature in this problem is the application to two bodies; notice 
that the treatment of each mass is made separate and distinct despite their 
obvious similarities , 

Sample Problem Solution 



CORRECT ANSWER: 3,27 m/sec 2 



The free-body diagrams for masses tnj and m 2 are shown below. 



Block of mass m 



1 



Block of mass m 2 



T 

A 



T 

A 



* 




7 

fl 
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Sample Problem Soluti on (continued ) 



Since the two blocks are connected by the same string, the tension is 
the same in both diagrams. 



mass m j mass m 2 



ZF - 

X 


0 


EF * 

X 


0 




EF y ‘ 


T — m i g * ttija 


EF y - 


T - tin - 





Note that the magnitude of the acceleration is the same for both masses 

and since m 2 > , m 2 has acceleration in the negative y-direction and 

m 1 has acceleration in the positive y-direction. 

Solving the simultaneous equations for a one obtains: 



a 



(m 2 - m 2 )g 
m 2 + m 2 



3.27 m/sec^ 



The emphasis in solutions i's on the correct answe r and why it is 
the correct answer. This has been shown to be more useful than identifi- 
cation of a wrong answer as wrong with additional discussion of why the 
choice was not acceptable. In some cases highly probable errors should 
be discussed, contrasting the incorrect with the correct answer, but the 
most useful kind of information is clarification of the correct answer. 
Therefore, prepare the discussions of the correct answers and add discus- 
sions of highly probable errors only if you have considerable confidence — 
aad data when possible — to indicate the nature of highly probable errors 
and misinterpretations, 

(ii) Enabling Problems -- The most useful aid for developing enab- 
ling problems is a well written solution to the corresponding core prob- 
lem, Each major step it. displayed in the core solution and these steps 
are, by definition, enabling objectives. 
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The enabling problems should be constructed to include the same 
physical principles and mathematical m ilpulation required by the associ- 
ated core solution step* It is not necessary or desirable, however, to 
make every enabling problem an undisguised fragment of the core problem. 
Clearly, one does not want to encourage mere parroting u 4 the core problem 
solution * 

Constructing problems which are parallel but not identical to others 
did not seem to present any special difficulties to course writers, although 
the process was as time consuming as creating a problem without any model. 
The 'ey elements in writing such parallel or equivalent problems are to 
include all the recently introduced scientific principles, methods, or 
facts which are contained in t e model, and to make sure the complexity 
level (see "Classifying Learning Objectives" in this report, and TR 5.C) 
is the same as that oi the model. 

Most writers found it easy to maintain complexity level even when 
they did not learn the precise category definitions* They simply made 
sure that the solution of the equivalent problem required the same 
symbolic equations as the original problem. This method autorru. ~ally 
insured corresponding complexity levels. It is important to keep the 
complexity levels of enabling problems at or below the level of the 
associated core problem* 

A sample sequence of core and enabling problems is presented here. 

The solution to the core problem (21) has three key elements: the 

identification of all forces acting on the block, the resolution of the 
forces into x- and y- components, and the solution of a set of two, linear 
equations. These are treated respectively by enabling problems 22, 24, 
and 23, The sequencing does not follow the same order as the steps in the 

O ^4 





core problem solution because problem 23 is an enabling problem for 24. 
Thus, the sequencing is quite flexible in an enabling sequence. 

Sample Core and Enabling Sequence 



21. 4, force ?, as shown below, of 10 nt pushes a 3-kg block along a 

plane inclined at 30 e . If ? is parallel to the horizontal surf ce, 
calculate the value of the normal force on the block. 




22. A block is pulled along a horizontal, frictionless surface by a 
horizontal rope as shown below. Which of the following force diagrams 
applies to the block? 




7777777777/7777 77777777777777777 
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Sample Core and Enabling Sequence (continued) 



22. continued 



reaction 




reaction 



C. 



reaction 



weight 




tension 



D* 



reaction ^ |r 

weight 



23. Moat physics problems in our course result in the formulation of 
equations of one or more unknowns. We know that the physics of the 
problem is complete when there are as many independent equations as 
there are unknowns. Find y for the following set of equations: 

lOx 4 - 2Qy ~ 60 

% * 

5x - 45y - -80 
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Sample Core and Enabling Sequence (continued ) 



24, A force of 10 nt pushes a 3-kg block along horizontal friction— 
less table. What is the value of the force exerted hv the table on the 
block? 




A. 7.1 nt vertically up 

B. 36.5 nt vertically up 
C* zero 

D, 10 nt vertically down 



Solutions to the enabling problems must be given the same care as 
was given to the core problem solutions. Again , new material is t > be 
stressed and it is often helpful to restate new principles, etc., in full 
in the enabling problem solutions. The procedure is the same as was de- 
tailed for core problems in (i) above. 

(i i i) Develop Core Prime Problems — For each core problem, an 
equivalent core prime problem must be constructed. The elements of 
developing equivalent problems were discussed in (ii) . Moreover, the 
remarks pertaining to core problem development pertain to the core primes 
as well. Usually, core prime problems will appear rather similav to th 
original core with variations in the sought- for quantities. 
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(Iv) C onstruction of Problem and Solution Book — We found that a 
convenient unit of subdivision for the P&5 book is the segment consisting 
of topical units which correspond roughly to one- half week's conventional 
work, or a standard textbook chaptar. On the average, about five core 
problems are contained in a segment, although they range from three to 
seven per segment. 

The format of a segment can be regarded in two parts, the problem 
statement portion, and the scrambled solution t-ortion. Problem statements 
are listed in sequence with the appropriate information panels as in the 
following example: 





Information Panel 


10. 


Core Problem 


1 . 


Core Problem 




Information Panel 


2. 


Enabling Problem 


11. 


Core Problem 


3, 


Core Prime Problem 


12. 


Enabling Problem 




Information Panel 


13, 


Enabling Problem 


4. 


Core Problem 


14, 


Gore Prime Problem 


5. 


Enabling Problem 


15. 


Core Problem 


6. 


Enabling Problem 


16, 


Enabling Problem 


7. 


Enabling Problem 


17. 


Enabling Problem 


8. 


Enabling Problem 


18, 


Core Prime Problem 


9. 


Core Prime Problem 







Note that more than half of the core problems are preceded by an infer- 
nation panel and that core problem (10) has no associated enabling sequence. 
The core problems should be clearly identified t we enclosed each core 
problem in a rectangular box, sample page of problem statements is 
included here (it is only acci lental that the core problem appears at the 
top of the page) , 
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Sample Core Problem — In a one-dimensional elastic collision 



between two objects, mass is initially at rest. i-. the initial speed 

of mass m^ is u = 1 km/ sec , and - 2m^ , what is the final velocity of 

m ? 

1 



Sample Core Prime Problem — — A mass traveling in the positive 
^“direction collide^ head-on and perfectly elastically with a stationary 
mass * After the collision is moving toward the negative and 
toward the positive x-d i rection , both masses having the same speed. What 
is the value of the ratio /m^ ? 

The definition of problem equivalence, however, is broad enough to 
include problems which are superficially dissimilar. For example : 



Sample Core Problem — A 1/ 2-ounce bullet traveling horizontally 
with a muzzle speed of 1500 ft/sec strikes a 5-pound block suspended from 
a fixed point with 5-foot, massless, inextensible cord. If the bullet 
remains embedded in the block, how high does the block rise? (Neglect 
air resistance.) 

Sample Core Prime Fiuolem — A 10=gtn bullet traveling horizontally 
with a speed of 500 m/sec strikes a 3-kg wooden block resting on an icy 
surface, (Neglect the frictional force between the bio k and ice surface.) 
The bullet remains embedded in the block and the combined body strikes 
a spring with a spring constant k - 30 nt/rn. Find the maximum compression 
of the spring. 
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After the ordered sequence of problem statements, there folic vs a 
scrambled listing of problem solutions. In this case, scrambling means 
that the solutions are not put in the natural order ' presentation, but 
are listed at random. This avoids inadvertent disclosure by the student 
while he is reading the preceding solut ! on. Scrambling is easily achieved 
during the typing process when the draft is being made into hard copy. 

As the first step in scrambing, the typist chooses one of the solu- 
tions at random and labels this [a] , More than one solution will often 
fit on a page so the typist then chooses another solution which approx— 
Imat ~ly fills the page (on occasion, three or more solutions can be put 
on a single page) , The second and third solutions on the page are labeled 
fb] and [c], See the accompanying sample solution page. 

When a solution [a] extends beyond a single page, say page 18, and 
ai >ther solution can be fit be ow the continued portion of 18 [a], then 
the first 1 ’new 11 solution on the continued page (p. 19) is labeled 19 [a]. 
Naturally, a record must be kept of the locations of scrambled solu- 
tions, This function is served by an Item Location Sheet (see sample). 

Here each problem number has a solution location associated with it; eg,, 

■ % 

17b means page 17, solution [b] * The other entries on the Item Location 
Sheet are made later, 

V7n found that paginating by the segment gave a more flexible product 
for revision purposes* Most of the segments in the physics program have 
about thirty five printed pages* This is rather a small number to bind 
individually, so for ease in handling we gathered groups of three to five 
segments to form volumes . The individual segments within one volume are 
clearly separated by a colored Divider Sheet. 

Each volume must be bound so that the pages will lie flat when opened. 
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SAMPLE SOLUTION PAGE 



8 



SEGMENT IS 



[a] CORRECT ANSWER: 43 nt 

If m 2 is to start moving, the applied force F must be greater than the 
urn of frictional forces, i.e„ i 

P > £ 1 + f 2 

where 

f I ■ Wai N i " Psi®i8 
f 2 - P a 2 N 2 ■ v b 2 ( m l + ®2> 8 
Thus the minimum necessary force is 

F - fj *r r 2 - y e i®l8 + M 8 2<®1 + 8 

" 43.1 nt ■ 43 nt 



[b] CORRECT ANSWER: 10 nt 

FirBt, we treat the blocks as a group to find the acceleration; 

then we isolate the block of mass m to calculate the force of contact. 

Since the force of friction is ignored, the sum of the forces in the 

x-direction is 



5Zf x “ F - (mj + m 2 ) a 

Therefore 



a ™ 2 m/ see ^ 

Now for the block of mass m 2 to accelerate at 2 m/#ee 2 , an unbalanced 
force must act on it. Therefore, 

2Z F x “ F * ®2 a 

- 10 nt 

the force acting on mass m 2 . 
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This allows students to work from the P&5 book without having to hold 
their place. We found looseleaf binders to be satisfactory in early 
versions of the physics program, but we prefer plastic comb bindings 
with soft covers because they are easier to store and handle. 
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STUDY GUIDE 



The Study Guide is the instrument of internal management for the 
system. It provides the basic direction for the student’s progress 
through the learning material and leaves his progress record for eval- 
uation purposes. This wet-to-reveal sheet, separate from the presenta- 
tional material to increase its flexibility, contains reading assign— 
ments , homework assignments, and multimedia assignments that are re- 
lated to the objectives under study in the segment. 

The student reads the problem in the Problem and Solution Book , 
and responds to it on the Study Guide* By using a special pen he 
reveals the correct answer and the page where the problem solution is 
detailed. He can then go to that page, to a subordinate enabling prob- 
lem, or to the next core problem* 

The Study Guide structure is exceptionally flexible. It does not 
require that questions be framed in multiple-choice format alone; prob- 
lems can be of the completion type, or true/false as well, by leaving 
room for the student to actively respond to the problem below the wet- 
to-reveal box and have him compare his constructed response with the 
correct answer. This method, of course, requires the student to evalu- 
ate his own response, but since he is not penalized for being incorrect 
he will most probably be objectively critical. The emphasis here is on 
the correct answer and how the problem is solved, rather than the iden- 
tification of an incorrect answer as wrong. 

The Study Guide is flexible also in that it can be modified with- 
out any corresponding change to the Problem and Solution Book. 
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In an attempt to free instructor’s time for tutorial assistance. 



the earliest Study Guide was designed to be machine read. An IBM 
punch card was inserted into a port-a-puneh machine which had an elec- 
tronic grid under the surface. When the student chose an answer to the 
multiple-choice problem presented in the branching scrambled text, he 
recorded that selection by punching through the card with a probe, clos- 
ing the circuit and making a bulb light under an answer matrix. The 
lighted square silhouetted the page number to which the student was to 
turn for confirmation or remedial assistance for his particular answer 
choice. He then was directed either to return and select another re^ 
sponse to the same problem or to go on to the next problem. The punched 
card was collected at the end of the lesson, supplying information on 
the student ’s path of progress to the instructors and course designers* 

There were several drawbacks to this method. In ordar for the page 
number on the paper matrix to be visible when lighted from behind, it 
had to be carefully registered, printed densely, and covered from easy 
viewing. We could not both protect it and reveal it adequately. 

The punched cards had to be handled carefully to insure their 
readability by machine. Students are just not that meticulous* The 
punches had to be clean jabs, not gouged, or they would spill over into 
one another. 

The need of providing every student with electronic equipment for 
lengthy course work is obviously costly, and the machinery is less port- 
able and more space consuming than paper matrices with chemically re- 
vealed printing. 

The course items had to follow the multiple choice format, for 
there was no means of putting any written material on the tightly 




54 

60 



constructed punch card or into the paper matrix. Additional directives 
other than page indicators had to be contained within the scrambled text 

To obviate the electronic equipment and its shortcomings, a chem- 
ically imprinted matrix response sheet was devised. Page numbers indi- 
cating the associated feedback page were printed with wet- * o-reveal ink 
for each answer choice. By using a special pen a student revealed the 
page number where he would find feedback for his selection. In addition 
the special ink was made to be optically scanned by instructor and com- 
puter, since student progress was an important instrument of course 
evaluation . 

The multiple choice distractors in these formats all led to 
remedial instruction. The incorrect alternatives had been designed 
intuitively by the subject matter experts, with the intent of "second 
guessing" the students — often a difficult and inefficient effort. 
Analysis of the data indicated that many of the incorrect choices went 
uns elected , These time consuming discussions of incorrect answers were 
no more effective in bringing a student to crit rion than a full discus- 
sion of correct procedure. A great deal of attention to a wrong answer 
or procedure may even reinforce this incorrect behavior. 

These early Study Guides did not include homework assignments, 
additional reading, and directive to other media sequences. Because 
these were all part of the Problem and Solution Book, modification of 
these materials was impractical . Thus, students’ paths through the 
learning materials could not be easily effected. 
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Development 



Study Guides are developed by segment to correspond to the Problem 
and Solution Book* This insures flexibility in that whole topics may 
be omitted or revised without disrupting the Study Guide format of 
adjacent segments. On the other hand, a segment is likely to be the 
largest learning unit to be required by a student in one sitting so 
that only one Study Guide needs to be handled in a study session. 

Various items which have already been developed need to be assem- 
bled as part of the Study Guide construction. following must be 

collected corresponding to each core problem in the segment! 

Readings ~ Text material for the given objectives has already 
been selected, A reading assignment is enter on the sample Study 
Guide as 

j 

Reading : HR 10-6 

Audiovisuals — In the physics program, media were developed In 
paPaZlel ; that is, illustrated text, talking book, and videotape all 
treated the same topics in order to provide alternatives for student 
preferences. Thus, "COLLISIONS" is the title for each of these media 
and our sample Study Guide simply refers to 
Audiovisual, COLLISIONS 

without identifying a specific medium, A new project is more likely to 
have presentations which exist only in one of the media. An obvious 
modification might be 

Talking Book, COLLISIONS 
when only a talking book is available. 
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Information Panels — These usually summarise the terminal objec- 



tives In a form which should be understood by the student before he be- 
gins any intensive problem solving. As a consequence, the information 
panels will ordinarily be referenced just before core problems in the 
Study Guide, / 

Core & Enabling Sequence — Answer spaces in the Study Guide cor- 
respond to completion or multiple-choice items in the Problem Book, The 

correct answer to a completion problem is invisibly imprinted in a box 

1 

together with the Prob 1 Book location of the detailed solution, A 
space is provided just below the box for the student to write his answer 
before he reveals the latent image in the box. The revealed entry for 
a completion problem is shown below: 



1/3 km/sec (ll[a]) j 

C ans ) 

Multiple-choice items have invisible x 1 s imprinted in each of the 
incorrect answer selections. In the correct answer position, the loca- 
tion of the solution is imprinted. An example of this multiple- choice 
entry is,: 

A B C D 



Naturally, if the Problem and Solution Book is not scrambled, only 
a check is necessary to indicate the correct solution. The student will 
then simply proceed to the solution in the Problem and Solution Book as 
a matter of course. 
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The sample Study Guide shows boxes labeled T and F next to each 
core level entry. These were included as a policing device to indicate 
whether a student actually turned to the solution location. This was 
accomplished by including a very simple true-false question at the end 
of the solution; if the correct response to the question was not re- 
vealed on the Study Guide, it was a strong indication that the item was 
not read# As we indicate elsewhere, such monitoring is likely to be 
objectionable to students who are sufficiently mature to use a self- 
managed system. 

Homework Assignments -- Homework assignments will usually appear 
at the end of a segment because they should not interrupt a basic unit 
of instruction. 

Branching Instructions — - An instruction which directs the student 
to a place in the Study Guide, determined by performance on an item, is 
referred to as a branching instruction# An example of such an instruc- 
tion is found in the sample Study Guide: 

If correct, advance to P 6; if not, 

continue sequence # 

In this case , the student is told that if he has successfully com- 
pleted the core problem, he may go on to the next core problem. Other- 
wise, he should work through the enabling sequence. This basic branch- 
ing instruction occurs frequently# Other branching instructions can 
arise for other conditions: 

If you had difficulty with F 6, read HR 10-7. 

Otherwise, go to Step 9.1. 

After all the item references for the Study Guide have been collected, 
they must be sequenced# We found that a Sub— committee of two course 
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authors could establish an ordering which was generally satisfactory. 

The format of the Study Guide is evident from the sample. In the 
first column, the problem numbers from the Problem and Solution Book 
appear* Core problems are singled out by putting bars above and below 
their problem numbers. The second column is simply an identification 
number for instructions* We labeled according to the core problem that 
the instruction appears under; the first instruction under core problem 
6 is numbered 6.1, 

Alternate formats may be devised for Study Guides when a latent 
image process cannot be used* Perhaps the simplest is to put a code 
number next to each answer space so that after the question has been 
answered, the student may turn to a key sheet which provides the "re- 
vealed" information next to the code number* 
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U.S. NAVAL ACADEMY STUDY GUIDE SELF-PACED PHYSICS 




REMEDIATION 



At the outset of course developments remediation was considered to 
be a major component of individualized instruction. The importance of 
removing deficiencies before students moved on to new material is appar- 
ent in the various kinds of remediation provided in the physics course. 
The entire notion of remedial Instruction, and its function, design, and 
placement, as well as its relation to other course components, under- 
went considerable revision as course development proceeded, 

"Remedial" assumes student failure has identified the need for the 
remedial work! if remedial materials effectively provide the necessary 
instruction, then why shouldn’t those materials be moved to the front 
end o£ the course to prevent the failure from occurring at all? As a 
result, remedial materials have been redeployed and are presented as 
learning materials in advance of testing and failure. 

The most widely used remedial material is that which provides 
immediate feedback to the student as he progresses through the learning 
materials. In our earliest efforts the student’s response to a multiple- 
choice item, the specific dis tractor he chose, directed him to a page 
of remediation for that choice. His response was then reinforced if cor- 
rect; if incorrect he was either told why his answer was wrong or given 
a hint to shape his thinking, and then sent back to try the problem again. 

In that original format, an effort was made to include remedial 
instruction of this kind for each incorrect answer selection. The data 
indicated, however, that these time-consuming specific remedial frames 
were not more effective than one general remedial frame for all dis trac- 
tors , probably because of the difficulty in anticipating student errors. 



As a result, we eliminated the extensive remediation at the incorrect 
answer choice, and expanded the discussion for the correct response* 
Thus, we were able to reinforce the correct answer or procedure rather 
than emphasize an incorrect one. The extensive reinforcement is now 
available to all students, not only the percentage of those responding 
incorrectly * This eliminated a great deal of page turning, which the 
students and faculty all found objectionable, conserved student time 
in extraneous remediation, and eliminated the need for course writers 
to second guess students. It also permitted the use of other than 
tnul tip le— choice questions, since all variations of a completion problem 
cannot possibly be anticipated. 

Group remediation was also available to our students. One class 
hour a week was set aside for students to raise discussions and seek 
extra assistance. It was originally intended that students would be 
assigned to lecture sessions covering those topics in which groups of 
students were most deficient. The time pressures on the professor who 
could prepare a lecture only after student T s weekly worksheets were 
sorted made this a task difficult to implement. 

Instructors listed and maintained office hours for those students 
requiring and seeking tutorial assistance. Test and study guide result 
were analyzed by the computer and students having deficiencies were to 
meet with instructors to resolve problems before proceeding to new work 
Details of this session were maintained by the instructor for the stu- 
dent's progress file and for revision purposes* A sample of the Indi- 
vidual Remedial Session form is attached. Professors did not prove to 
be very reliable in bookkeeping, although they certainly served their 
tutorial function. As such, these records should not be heavily relied 



upon for revision purposes . 
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SAMPLE 



INDIVIDUALIZED REMEDIAL SESSION 



Date 

Tima 



Suggested Steps For Documentation 



Student Name 
TO Number 



(1) Retest of TO Question 



Success 


Fai lure 







(2) Ask student to state brie fly what he feels he doesn’t understand 
about: the TO Question (sv/nmarize briefly below) ■ 



(3) Check list of enabling objectives which might be covered to allow 
student to better understand TO Question. 

List of Course Objectives Treated Commen ts 



(4) 



Record briefly all additional personal inputs that may have been 
required to enable the student to achieve the TO Question . 





Success 


Failure 


(5) Retest of TO Question 
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In the study of physics, each topic builds upon certain ubiquitous 



topics and techniques, A failure on the part of a student to master 
any of these basics is likely to jeopardize his performance in suc- 
ceeding topics. To rectify deficiencies of this sort, we prepared short 
(one or two page) "packages", to be Inserted in the study guide where- 
ever and whenever their appearance might be called for. 

The emphasis in each of these interchangeable packages is on appli- 
cation; most of the titles begin "How To,.,", Each package contained 
an example followed by a question. The pragmatic approach was used 
because a student would choose these packages when he is primarily con- 
cerned with another topic. 

Although prepared, our experience with extraneous remediation kept 
us from including these packages in an experimental run. Parts of the 
f, How to" packages were incorporated into the problem solution. The 
surviving aspects of these packages were not at a sufficiently elemental 
level to serve the original purpose of providing a reminder to the stu- 
dent of certain prerequisite topics, especially in mathematics. We rec- 
conunend, however, that course developers consider reintroducing such a 
remediation component. 

Early in our course development we presented audiovisual material 
at the beginning of each frame together with the reading assignment and 
supp lementary notes. Later, when we expanded our distractor frames, 
many of the AVs were presented as remedial material in these remedial 
branches. As a consequence, only students who selected an incorrect 
answer viewed these remedial AVs, 

With our emphasis away from anticipatory remediation, many of the 
audiovisuals relating to distractors were eliminated. Those AVs which 
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SAVfPLE "HOW TO" SHEET 



How to Solve Linear Algebraic Equations 

Solutions for N unkno is may be obtained from N Independent equations. 
Using a substitution method, one can solve the first equation for any 
unknown and substitute the expression obtained In the second equation. 

The substitution process is then repeated using the second and third 
equations, and so on. Ultimately, only one equation with one unknown 
will remain. 

Example: 

Consider the following linear algebraic equations with unknowns rjS.t: 

(1) 7r - 5t ® -1 

(2) 3r + 5 * 7 

(3) r - 25 + t “ 3 

Note that there are 3 equations with 3 unknowns . 

By the prescription, we ’solve' equation (1) for any unknown, say r: 

r » (5t - 1) fl , 

and substitute this expression for r into equations (2) and (3)# Now 
solve the altered version of equation (2) for any remaining unknown, say s: 
s - 7 - 3 (5 t - 1) /7, 

and substitute this expression for s into (the updated version of) equation 

(3). Finally, only one unknown remains: t. We obtain t * 3, This result 
may be substituted back into any two of the original equations, and the 
whole process may be repeated to find the remaining unknowns * 

Find x for the following set of equations (a,b,c,d,e are considered 
•known’) : 

a x + b y ® 0 
c x + d y ■ e 

I 

a) ad / (be - ed) 

b) ad / be 

c) eb / cd 

d) eb / (cb - ad) 



Please return to Question 



on page 



did not specifically address errors were moved forward as an additional 
source of information. 

An additional remedial technique was found in the original review 
or M interest! tial" segments, These aids were additional dis tractors 
called Help 1 and Help 2. Help 1 provided the student with a hint on 
solving the problem, A simple note like ’'Three forces are in effect 
here : weight , tension ? and electrostatic repulsion” would provide assist- 
ance in starting the calculation. If after reading Help 1 he was still 
unable to complete the problem. Help 2 would provide additional instruct 
tions , a more complete ,f waik through 11 of the problem solving procedure. 
These Helps were well received by the students and faculty, and ulti s 
mutely evolved into our full solution statement. 

A catch-all formal remedial sheet is used for remediation after 
testing. These have the form shown in the sample remedial sheet. It 
restates the core problem associated with a problem missed by the stu- 
dent on the test. It serves as a reminder of the broad objective and 
orients him so as to put his deficiency into proper context. The stu- 
dent may then return to the course materials that relate to that problem. 

To further supplement the course materials, self study learning 
packages, relevant to course topics, were assembled. References to these 
were provided in the formal remedial sheets. 
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SAMPLE HELP 1 



7 - 2 - 4. 3 



page 96 



One simple way to solve this problem is to consider what happens 
physically. The shells are fired and the boat recoils with some velocity. 

The velocity of the recoiling boat is very much less thail the velocity of 

the mass of the boat. We need to know the total mass of the fired shells in 

order to calculate the total momentum of the shells fired. Since the velocity 

of the boat is small the muzzle velocity ma) be taken as the velocity in 

the inertial frame initially at rest with respect to the boat. Application 

at this point of the principle of conservation of momentum gives the 

result. 



Please return to page 59 and select an answer to Question 3-4, 
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7-2-4 , 4 



SAMPLE HELP 2 



page 100 



Let's use the principle of conservation of momentum to 

solve this one. 

The velocity of the boat is small compared with the muzzle velocity 



and the mass of the fired shells : 
the boat. The muzzle velocity is 
to the moving boat. We would not. 
more than several feet per second 
muzzle velocity is the same as the 
stationary ground we cannot make c 
fix*ed in 5 seconds is 

x 600 x i x 2 x i x i x sec l 
60 16 32 



is very much smaller than the mass of 
the velocity of the shell relative 
expect the boat to recoil with any 
so that if we assume that the 
5 velocity of the shell relative to 
i large error. The mass of shells 

, shells „ min .. oz .. s lug _ 
mi n sec snell lb 



100 

(16) (32) s iugs. 



Momentum is - x 3200 = }L. slug ft 'sec. This is equal to 

t JLb ) [32 ) ( 16 ) 



the momentum of the boat which is 2000 v from which we obtain v. 

32 

Notice that we neglected the mass of the fired shells in considering 
the mass of the boat. 



Please return to page 59 and punch the correct answer to 
Question 3-4. 
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SAMPLE REMEDIAL SHEET 



SEGMENT 3 

MOTION IN ONE AND TWO BX1VENBIQN5 
Problem 9: Motion in a Horizontal Direction 



9, 




The distance between point A and 
wall B is 3000 ft, A car can 
develop a maximum acceleration of 
15 ft/sec 2 . The maximum deceleration 
that the brakes can provide is 
30 ft/see 2 , The driver of the car 
wants to reach the wall B in the 
shortest possible time, starting 
from rest at point A, He uses 
the full accelerating capacity of 
the car. What is the shortest 
distance from B at which he must 
apply the brakes if he is to avoid 
crashing into the wall? 



Reading Assignment; 



Hallidfty and Resnick; 


Ch. 3 j 


Sect , 


, 8 








Semat and Blumenthal; 


Vol. I. 


, Ch. 


3, 


Fr 


26 . 


-28 


Joseph and Leahy: 


Part I 


, Ch. 


2, 


Sect, 


8, 




Related Problems: 












Schaum: 


Ch. 4, 


Nos . 


1, 


2, 


5 ? 


6 
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AUDIOVISUAL COMPONENTS 



The audiovisual components available in the present program include 
videotapes, talking books, illustrated texts, and lecture /demons t rat ion 
classes. Videotapes were used it preference to film simply because of 
convenience and accessibility at both NYIT and the Academy* Talking books 
contain pictures and diagrams accompanied by a "voice-over 11 discussion 
from a portable cassette tape recorder. The illustrated texts have the 
same picture format as the talking hooks, but the discussion is provided 
in print opposite the associated figures* A conventional classroom 
lecture with demonstrations and blackboard work constitutes the lecture/ 
demonstration* 

Experience with the physics program indicates, perhaps surprisingly, 
that audiovisual materials are not essential components of a self-paced 
program for highly motivated college students of hard science. Our 
Investigators sought to determine the effectiveness of the various audio- 
visual components for students with known background variables. No sign!* 
f leant differences in performance due to audiovisual materials were 
detected! However, individual students do have preferences among audio— 
visual components, and this is the basis of the present system of voluntary 
selection. 

Although the use of audiovisual materials evidenced no significant 
performances in the operating program (see 5.0, Statistics: 1969 Experi- 

ment), we must not conclude that they are without significant value* We 
believe these materials are useful for teaching "trouble topics" which are 
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Qj_ ten difficult for students to conceptualise . These often require visual 
or auditory amplifications redundancy and repetition in various perspec- 
tives s and motion. Trouble topics had little representation on the Academy ! s 
examinations even though some instructors insisted that they were essential to 
the course content. This disparity between what a student ought to know (as 
revealed in course objectives) and what he is expected to know (as revealed 
in testing) is apparently widespread in the community of science educators. 

For many professors, it may be that conscience dictates the inclusion of 
some difficult topics in the course— —later , conscience dictates that these 
topics be excluded from examinations . 

Audiovisual materials are also used to provide overviews, enrichment, 
and to demonstrate the relevance of subject matter in real-world settings 
and real-life applications. It seems that these are not significant to the 
task of achieving the specific bahavioral objectives. However, such presen- 
tations are likely to stimulate audience interest in the subject matter — -a 
very desirable general objective for any course. It is improbable that audio- 
visuals increased interest or motivation in Academy midshipmen ; they have no 
time for such luxuries. Other less motivated and less goal-directed student < 
groups, however, may profit more from audiovisuals. 

Three distinct weaknesses of audiovisual material for a majority of 
college physics topics are now evident: they are inefficient and time-con- 

suming sources of information (as compared with textbooks, for example); they 
allow for student interaction only with some difficulty and awkwardness (es- 
pecially notable for high level objectives) ; and they are not easy to ,f skim t " 
so that locating a particular fact or objective may require extensive searching. 
These weaknesses suggest that audiovisuals should not be recommended casually 
for run-of-the-mill ob j ectives—even when cost is not an important consideration 
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In the initial phase of the program, 48 videotapes were developed to 
provide broad coverage of major course topics. The objectives to be treated 
were chosen primarily on the basis of "fundamental importance" to physics * 
Our investigators reasoned that more precise selection criteria for audio- 
visuals would evolve fro^ the results of this shotgun approach. 

No attempt was made to produce professional looking audiovisuals in 
this phase— -the search for some gross selection parameters did not justify 
expensive refinements. Videotapes averaged 15 or 20 minutes each, and 
active responding was provided by presenting a question on each tape. 
Frequently, questions requiring only a verbal response were asked by a 
narrator who then paused before giving the answer. Various videotape 
formats were used including blackboard lectures, laboratory demonstrations 
with voice-over narration, animation, and demonstrations in real-life 
situations . 

When no performance differences were found to be attributable to 
audiovisual usage, a new set of videotapes was developed according to 
a more stringent list of selection criteria. Our psychologists and 
subject specialists expected that by basing audiovisual development on 
those factors which were anticipated as most likely to enhance conven- 
tional instruction, the influence on performance would be more pronounced. 
The audiovisual selection criteria for the revised program are as follows: 

a# Student difficulty as determined by past experiences in conven- 
tional courses 

b. Need for motion (videotape only) or serialization of motion 
f ^ top motion) 

c. Concrete demonstration of abstractions and concepts 
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d . Imitative guidance whereby students see how an instructor 
approaches a problem 

e. Pictorial media needed for situation not easily verbalized 

f . Importance and relevance to the remainder of the course 

Ail course topics were assessed according to whether each criterion 
was unimportant (blank) , important (*) , or very important (**) , as shown 
in the sample page Audiovisual Selection Gvitevia . Any topic assigned 
two or more asterisks was given a videotape pres en t at ion-*twan ty-s ix in 
all. The list of criteria has no entry called "need for overview" which 
we now believe may be one of the most useful roles for audiovisual materials. 

The production quality of videotapes was improved in the revision, but 
high professional veneer was still not Justified or sought. A check on the 
influence of quality was made by developing two versions of the videotape 
Kivdhhof f ( s Rules with one having much more polish. Not only T ere there 
no discernable performance differences, but students’ preference ratings 
ware also indistinguishable for both versions, 

A talking book and illustrated text were developed in parallel with 
each videotape. The information content is the same for the parallel 
videotape, talking book, and illustrated text. 



Audiovisual Development 

After a topic is chosen for audiovisual treatment, a storyboard must 
be developed. This is a series of sketches with descriptions depicting 
scenes and pictorial action in a kind of comic strip format. As usual , 
it helps considerably to begin with a clear statement of the objective. 

By using rough sketches and a few written remarks directed at attainment 
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SAMPLE 



AUDIOVISUAL SELECTION CRITERIA 





Title 




Criterion 


Guideline 








a 


b c_ 


jd _e 




1. 


Projectile Motion 




* * 




k 


2, 


Newton T s 1st Law 




* * 




kk 


3, 


Newton’s 2nd Law 




* * 




kk 


4. 


Newton’s 3rd Law 




* * 




kk 


5. 


Atwood ’ s Machine 


* 


•k 


k 


k 


6. 


Centripetal Force 


* 


k k 


k 


k 


7. 


Work When Force Varies in 
Both Magnitude & Direction 




k 




k 


8. 


Potential Energy 




k 


k 


k 


9. 


Kinetic Energy 




k k 


k 


k 


10, 


Conservation of Energy 




k 




kk 


11, 


Conservation of Momentum 




k 


k 


kk 


11a. 


Impulse and Momentum 


* 


k k 






12, 


Collisions 


* 


k 


k 




13. 


Coulomb’s Law 




k 


k 


kk 


14. 


Calculations of E 


k 




k k 




15, 


Flux 


** 


k 


k k 


kk 


16, 


Calculation of E 
Using Gauss’ Law 


k 




k 


k 


17. 


Capacitors 




k 


k 


k 


18. 


Motion of a Particle 
in an Electric Field 




k 


k k 


k 


19, 


Kirchhoff’s Rules 


k 




k 


k 


20, 


Definition of B Field 


k 


k k 


k 


k 
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AUDIOVISUAL SELECTION CRITERIA 



Title Criterion Guideline 







a 


b 


c_ d_ 


f 


21, 


The Force Between Parallel 
Current-Car ^ ing Conductors 


* 




* * * 


** 


22, 


Ampere’s Law 


* 




* 


* 


** 


23, 


The Biot-Savart Law 


* 




* 


* 


** 


24, 


Faraday 1 s & Lenz 1 Law 


* 


* 


* 


* 


** 


25, 


Motions of a Charged Particle 
in Crossed E and B Fields 


* 


* 


* 


* 


* 
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of the objective, a storyboard is developed. We exemplify this for 

Movement of Center of Mass. 

The stated objective in this case is to exhibit that for every system 
there exists a point, the center of mass, which moves as though it were a 
particle with mass equal to the mass of the system subject to a force 
equal to the total of all external forces on the system. A storyboard 
directed at this objective is shown on the following page. 

Illustrated texts can be prepared from the storyboard by amplifying 
each frame into a full page figure and by writing the associated descrip- 
tive text on a page opposite the figure. In this way page turning is 
eliminated. The illustrated text for Movement of the Center of Mass is 
reproduced in full , 

Talking books have the same format as the illustrated texts but with 
the textual material replaced with a "voice-over" narration supplied by 
a tape recorder. We found that simply reading the illustrated text onto 
tape did not provide an acceptable narration. The printed word is more 
formal than the spoken word, and reading the printed material aloud gave 
a stilted effect. The text must be slightly reworked to less formal and 
more colloquial language in order to serve as voice-over narration. 

Developing videotape is an art and a good product requires the 
experience of a professional. We employed the services of a Media Di- 
rector who orked with the subject matter specialists to develop the 
finished videotapes. The basic ingredients to be supplied by the subject 
matter people are the storyboards and guidance in the preparation of 
narratives . 
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SAMPLE STORYBOARD 

Move e>F OF 
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MOVEMENT OF 
CENTER OF MASS 
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ILLUSTRATED TEXT 



U> 



Fig. 1 



The center of mass of an object may be described as that single point at 
which all of its mass appears to act. For an object of uniform density 
having some regular shape, such as a solid wooden ball, Its center of 
mass is easily located to be at the geometric center, as you can see in 
Figure 1* Finding the location of the center of mass for a hollow 
rubber ball is no more difficult — it too is at the geometric center, 
even though none of the actual mass of the ball is located at that very 
point. 

Many objects, having either regular or irregular shapes, have centers of 
mass located in space — probably the chair you are sitting on at this 
moment or the cup or glass you used this morning are good examples to 
consider. For these objects, the center of mass acts in every way just 
as it does for one having a center of mass within the medium itself — as 
with the solid ball. 
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CENTER OF MASS 



(a) for a solid ball 




(b) for a hollow ball 





ILLUSTRATED TEXT 



( 2 ) 



Fig * 2 



The concept of center of mass can be a powerful tool in the study of 
motion, since all rigid bodies, regardless of shape, volume, or density, 
can be considered to be point masses acted upon by external forces, 
thereby simplifying the application of Newton’s laws of motion. 

A task that at first seems difficult is the analysis of the motion of 

a body when internal forces are also acting. Let’s see what effect, 

if any, they might have. To do this, let’s examine the effect of an 

explosion on the center of mass of a system consisting of two equal 

masses* In Figure 2, you see two identical cars about to be exploded 

apart by a compressed spring- Before the explosion, the center of mass 

of the system is midway between the cars - When the explosion occurs, 

each car receives an equal, but opposite force to the othe- , for the 

same period of time, giving each similar accelerations But at any time, : 

the center of mass of the system can be found to be at t -ame point, 

unaffected by the explosion. 

j 

i 



i 

4 

i 
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EQUAL MASS CARS 



CENTER OF MASS 






CENTER OF MASS 
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FIGURE 



ILLUSTRATED TEXT 



(3) 



Fig. 3 



You may wp.11 ask, what would have happened if two unequal masses were 
chosen? Let’s repeat the explosion, this time with unequal cars; say 
they have a mass ratio between them of 1:2. Once again the explosion 
will apply equal and opposite forces on the cars, but this time one car, 
the lighter one, will accelerate at twice that of the heavy car, thereby 
moving twice as far in equal time. Consequently, the center of mass of 
the system remains in the same position, unaffected by internal forces 
as you can see by examining Figure 3* As a matter of fact, even if the 
two cars have some initial velocity while linked together, their center 
of mass would continue to move at that velocity even after the explosion 
occurs , 
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UNEQUAL MASS CARS 



CENTER 
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MASS 








CENTER OF MASS 

o 






l - ^ T — ✓ 

2x' x ' 



o 

ERIC 



FIGURE ® 
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ILLUSTRATED TEXT 



(4) 



Fig. 4 



Before closing, let’s apply these principles to some typical motion 
problem. A good one to consider would be the motion of an explodable 
ball as it moves in a parabolic trajectory. Here, In Figure 4, the ball 
is subjected to some initial accelerating force, and a constant gravita- 
tion force, both acting externally, as well as an internal explosive force 

Before the explosion the ball travels intact along a parabolic path 
governed by the effects of its initial velocity and gravitation* The 
ball is then exploded into fragments * each moving away from tbe center 
of gravity at a rate dependent upon the explosive force and its si^e, 
and each still is affected by the initial velocity and gravitation* 

Since the explosive internal force has been shown to have no effect on 
the center of gravity, its motion continues along the parabolic trajectory 
as though the ball had remained intact* 




FIGURE 
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HOMEWORK AND READING ASSIGNMENTS 



A homework assignment is traditionally a requirement which must be 
satisfied outside the classroom rather than one which may be satisfied 
at home. This distinction blurs in the case of a highly automated course 
with portable components ■ Except for videotapes and tests , all com- 
ponents of the program can be carried off campus and administered at 
home. The institution decides on the mobility of the components; the 
Naval Academy prefers that only the textbook be available outside the 



classroom. 

Originally, we felt bound to have traditional homework assignments 
which, for hard sciences, means a reading assignment and some problems 
from the appropriate textbook chapter . Four textbooks are used in the 
physics program: Hal 1 .i day and Resnick, Sears and Zcmansky , Baez, and 

Shortley and Williams. An objection was raised that too much redundant 
reading was being assigned from the various texts. It was decided that 
in the first revision of the program one textbook, Halliday and Resnick, 
should be used as the standard text and the others would be used as 
supplements . Sections of essential readings were indicated by an 
asterisk and the supplementary readings were listed in preferential 



order: 



ASSIGNMENT 



Heading 

HR 16-7*, 8*, 9* 
SZ 7-4 



4-1, 2, 3, 5 , 



9; 27-4 



Homework 



HR 16-13 
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At the same time we made a fixed distinction between reading and homework 
assignments. Homework refers only to problem assignments. 

This assignment fori, is maintained in the second revision of the 
program except that readings and homework now appear in the Study Guide 
rather than the Problem and Solution Book, The present format is shown 
in the sample Study Guide on the next page. The reading assignment is 
the first item and the homework assignment is the last item on the sample 
We believe that two more modifications should be incorporated into 
a new program. Firsts since the course is now based upon a complete set 
of all-encompassing core problems, all of which appear in the Problem 
and Solution Book, the homework problems must be redundant or superfluous 
On this basis we recommend emitting homework problems# Secondly, the 
reading assignments should be annotated with a short phrase or two to 
indicate to the student what it is he is about to read. Phrases such 
as "For an overview of electromagnetic theory see,*,," !l For an example 
of how a radio wave is created see,.. j" and "Philosophical implications 
of the Second Law of Thermodynamics appear in,*," have the virtues of 
acting as advance organizers and informing the student of those optional 
readings which are important or interesting to him. 

Choosing the reading assignments by working backward from the 
objectives assures that the primary readings are germane to the objec- 
tives . One must be careful, however, to look beyond those readings 
which directly enable the student to solve the problem at hand. Secon^ 
dary readings should include overviews and cultural and historical 
information pertaining to the objective in a broad sense* 



U.S, NAVAL ACADEMY 



STUDY GUIDE 



1 of 1 

SELF-PACED PHYSICS 



STEP 



0.1 



0.2 



1.1 



NAME 



Reading; *HR 16-9/16-12 
SZ 7-4 

Information Panel, "Gravitational 
Potential Energy" 



A B C D 



T F 



5,1 



6.1 



□□□□ □□ 

If your first choice was correct, 
advance to 5.1; if not* con- 
tinue sequence. 



A B C D 

□ □□□ 

A B C D 

□ □□ 



STEP 



A B C D 



□ □□ 



A B C D 



T F 



□ □□□ □□ 

Information Panel, "Gravitational 
Potent lal" 



A B C D 



T F 



□ □□□ □□ 

If your first choice was correct, 
advance to P 10; if not, con- 
tinue sequence* 



BCD 



□ □□□ 



ERIC 



(ans) 



10 



11 



12 



13 



14 



15 



16 



SECTION 



SEGMENT {4 



□□ 



(ans) 



10.1 



13.1 



14.1 



16.1 



□□ 



(ans) 



If correct, advance to 13*1; if 
not, continue sequence* 

A B C P 

□□□□ 

A B C D 

□ □□□ 

□□ 

(ans) 



Information Panel, "Escape 

T F 



□□ 



If correct, advance to 16. L: 
not, continue sequence. 



(ans) 



T F 



□□I 



(ans) 



Homework; HR 16-24, 16-25 
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ENRICHMENT 



Enrichment materials for the physics program are packaged in one 
Problem and Solution volume. This enrichment volume characteristically 
comprises optional problems which are either quite difficult or which 
cover subtle points. The outstanding fact to be told about the enrich” 
ment volume is that no student has ever elected to take it! 

The original arrangement of enrichment materials seemed more natural. 
Individual segments concluded with the enrichment problems and instruction 
pertinent to that topic . In this way the natural order of progressive dif- 
ficulty was maintained throughout the segment. Enrichment was distinguished 
by a statement that the material to follow was optional. However, when the 
data revealed that students were not doing the optional work, it was decided 
for pragmatic and psychological reasons to collect the enrichment sections 
together under separate cover. 

Because few students elect to try enrichment exercises, fewer copies 
of the enrichment volume needed to be printed and stored in this more optional 
packaging structure. More important, the presence of such additional ma- 
terial seemed to intimidate students who were already having great diffi- 
culty in completing the requirements. 

In view of the disuse of enrichment material, we examine the purpose 
of such material. Is it to provide a springboard to further study for very 
interested students? Is it to establish a superior grade for superior 
students? 

Serious practical considerations discourage the creation of enrichment 
material for interested students to use as a step toward further study, 

O 90 
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The high costs of development and production of these materials * and the 
very limited audience for them, make enrichment materials highly cost- 
ineffective items for this purpose. We think that this need can be ful- 
fil 1 ed by providing references to articles in such popular journals as 
Scientific American and Physics Today (other disciplines usually have 
corresponding counterparts) and to intermediate textbooks. Of course, 
in cases where seif-paced material has already been developed for higher 
level courses, these can be taken over directly for enrichment. 

The concept of using enrichment material for the purpose of 
establishing a superior grade deserves the consideration of those 
implementing the course# The successful completion of all "standard" 
material might establish a grade of 1 f B 1 * and the completion of additional 
enrichment material might establish an "A." In using graduated learning 
material to establish grades, the material should appear as a continuous 
whole# Only a note in the Study Guide is necessary to flag that "the 
objectives to follow are required for an "A" grade# 
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REVIEW 



The original review function was served by several interstitial 
segments which were distributed throughout the Problem and Solution 
Books. These provided drill and practice on the preceding two or three 
volumes of material. The format of these segments was similar to the 
regular segments, with the exception that two additional distractors 
were available for each review problem: Help 1 and Help 2. 

Help 1 provided the student with a hint on solving the problem. 

A simple note like "Three forces are in effect here: weight, tension, 

and electrostatic repulsion" would provide assistance in starting the 
calculation. If after reading Help 1 he was still unable to complete 
the problem, Help 2 would provide additional instructions, a more com-* 
plete "walk through" of the problem solving procedure. These Helps were 
well received by the students and faculty, and ultimately evolved into 
our full solution statement. 

Students complained that review material relating back as far as 
1-1/2 months disrupted their concentration and forced them to review 
material not covered by the next posttest. It was suggested that inter- 
stitial segments be rescheduled or eliminated. 

The drill and practice function of the interstitials were, in the 
later versions, collected together to eliminate the disrupting aspect. 

To provide review for midterm and final examinations, six review segments 
were prepared, three available before the midterm and three before the 
final. 

This present format is conducive to review whereas earlier versions 
were not. The Information Panels and problems in the Problem and Solition 
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Book are not scrambled, allowing the student to look through the mater- 
ial in a straightforward manner, as he would in any text. Previously ? 
students insisted on the addition of an index for each volume in order 
to locate specific topics. 
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TESTING 
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TEST ITEMS 



The development of test materials is greatly facilitated by having 
objectives cast in problem form, A test item is simply constructed as a 
perturbation of the corresponding core problem. This procedure usually 
results in 1 test item at the same complexity level as the core. 

In the physics program, it was not possible to test every terminal 
objective at the core pv<: blent level because, for some topics, this would 
require excessively long test sessions * As a compromise, we tested some 
objectives at the level below that of the core. These were easily con- 
structed as variations of enabling problems. 

Developing Problems 

Subject specialists usually have a wide repertoire of problems. 
Nevertheless, they are frequently called upon to construct "new" problems. 
It is our experience that this is usually expedited by a procedure of 
forming an idealized mathematical problem and then putting l, flesh M on the 
skeletal elements. Four basic steps comprise the procedures 

(i) Write the applicable mathematical relations for an idealized case; 
that is, consider massive particles rather than real objects, 
abstract fields rather than their causative agents, perfect heat 
reservoirs and sinks rather than approximate physical counter- 
parts, etc, 

(ii) Choose a suitable "unknown" quantity and affect a solution of the 
equations by inventing some simplifying conditions and "given 11 
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information . 



(ill) Write an idealized mathematical problem for the situation de- 
scribed above. 

(iv) Convert this mathematical problem into a specific physical problem 
by substituting real objects and situations for the idealized 
counterparts . 

As an example, consider the creation of a core problem on applying 
the conservation of energy theorem to an object moving in a uniform 
gravitational field. Any physicist recognizes the idealized situation t 
A point particle of mass m which has respective initial height and speed 
y 0 and v Q , and a final height and speed of yf cmd Vf obeys the principle 
of conservation of mechanical energy near the surface of the Earth, 



In order to make a mathematical problem out of this equation, one 
must choose a suitable unknown, say yf - y Q . The other items, v Q and Vf , 
must then be given or implied by the problem statement. Mathematical 
computation can be reduced at this point by judicious choices for these 
given items; one ^uch arbitrary choice in set Vf enual to some 
fraction of v c ? 




( 1 ) 



1 

vf = 2 v o 



( 2 ) 



With this simplification, Eq . (1) has the solution 



yf - y Q ” v o 2/4 8 



An idealized mathematical problem which incorporates these 



elements is : 
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A particle moves near the Earth f s surface initially at speed v Q . 

At a "final" point , the particle moves with speed v Q /2 * VThat 
is the increase in height of the final point over the initial 
point? 

Notice that the mass of the particle is not included since it is not 
required for the problem solution. 

The sterile idealized problem can now he converted to a more physical 
statement by changing the idealized objects and situations to specifics. 

By substituting a roller coaster for the idealized particle, we arrived 
at the following problem: 

A roller coaster moves at point A with speed v Q . At point B, 
the coaster moves with speed (1/2) v Q . Assuming no frictional 
losses, what is the height of point B above point A? 



Any number of similar problems may be generated in this way from a 
formal problem skeleton. In the present example ^ another substitution for 
the particle can be a pendulum bob: 

A simple pendulum swings through its lowest point at a speed 
of 8 ft/sec. What is the height above the lowest point when 
the pendulum bob has a speed of 4 ft /sec? 

Of course, the procedure recommended here is not the only way to 
create good problems. It is simply a categorization of the steps taken 
most often by our most experienced course writers. 



A. 3 v Q 2 /8g 

B, 7 v Q 2 /8g 
C- v Q 2 /4g 
D, 5 v Q 2 /8g 
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Usually, the writers 



did not trouble to write out an idealized version of the problem * but they 
agree that it was kept in mind while they transformed to the actual problem. 

Elements of Problem Writing 

Writing good problems is as elusive and difficult as any technical 
writing. A few important elements, however, can be watched for by the 
problem writer and editor. 

Most obvious is the fact that problem statement should be correct 
and the problem should be solvable. Numerical errors which imply that 
a satellite revolves around the Earth below sea level, or stated condi- 
tions which prevent a baseball from reaching the point at which its speed 
is sought, are very distressing to students and instructors. Most of our 
authors have committed such errors on occasion. The writer’s best insur- 
ance against such embarrassment is to have a second person check the 
problem, or to work in pairs. 

Each problem statement must be self-standing in that all symbols 
are defined (except on occasion constants G, e , h, c, and g) and sufficient 
context is provided so that the problem would be clear even in isolation. 

The following example does not meet this requirement: 

Two particles of mass 2 kg and 3 kg, respectively, are moving 
with a speed of 10 m/sec due east. A third particle of mass 
2 kg is moving with a speed of 25 m/sec due north. Find P/M # 



A, 


10, 


.1 


m/sec 


at 


45° 


N 


of 


E 


B. 


20, 


.2 


m/sec 


at 


37° 


N 


of 


E 


C, 


10, 


.1 


m/ sec 


at 


37° 


N 


of 


E 


D. 


20, 


,2 


m/sec 


at 


45 8 


N 


of 


E 
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Even though the symbols P and M are used extensively in the study of 
center of mass motion, this problem is wide open for misinterpretation. 
One may question the intent behind the statement; do we want the student 
to make an enduring identification of P and M as total momentum and total 
mass , or do we want him to be able to find the center of mass velocity 
regardless of the symbolism he chooses? The problem was changed to: 

Two particles of mass 2 kg and 3 kg, respectively, are moving 
with a 3peed of 10 m/sec due east, A third particle of mass 
2 kg is moving with a speed of 25 m/sec due north. Determine 
the velocity of the center of mass, v cm , °f the system of 
three particles* 



A* 


10,1 


m/sec 


at 


45 e 


N 


of 


E 


B, 


20.2 


m/sec 


at 


37° 


N 


of 


E 


C. 


10.1 


m/sec 


at 


37° 


N 


of 


E 


D. 


20.2 


m/ sec 


at 


45° 


N 


of 


E 



Occasionally, one problem will refer to another for information or 
results* This practice should be kept to a minimum, and only when the 
problems will always appear as j. u of the same sequence* 

Ambiguity in problem statements is a most insidious trouble to guard 
against* Ambiguous statements can occur in several ways. Carelessness, 
unstated assumptions, and words with both popular and technical meanings 
are the most frequent causes of ambiguity which we have experienced* 

Some examples will help to illustrate these* 

Many questions have more than one correct answer, but the multiple 
choice format allows only the desired response to appear. Due to care- 
lessness, a second correct answer sometimes appears in the list of 
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distractors. One final examination question read as follows. 

The physical quantities of length, mass and time in the 

MKS system are; 

A, derived quantities. 

B. standard quantities. 

G. relatixe quantities. 

B. fundamental quantities. 

E. none of the above , 

In this case, both B and D are correct. Naturally, no short answer 
or completion problem should have more than one correct answer. 

Unstated assumptions often contribute to ambiguity in a problem 
statement, and they are not easily detected because most subject experts 
share the same tacit assumptions. In the following problem, no state- 
ment is made regarding whether or not the plane has friction, although 
this makes a considerable difference in the execution of the problem; 

A block of mass H is released from rest on an inclined 

plane of angle ©. Find an expression for the acceleration 

of the block. 

Of course other acceptable tacit assumptions are contained in this prob^ 
lem, It is assumed that the system is near the surface of the Earth, 
for instance. This is, nevertheless, a discrimination which the student 
is expected to make from the problem context. 

Another example from a final examination illustrates an ambiguity 
which can arise as the result of an inadequate distinction between 
popular and technical word definitions: 
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A rocket that weighs 5000 lb. on earth is in free ™ace and 

fires a small course correction motor, generating a thrust 

of 60 lb. What acceleration results? 

A. 1.2 x 10 -2 f t/sec 

B. 38,4 x 10“ 2 ft/s6c 

C. 1 .92 x I0 2 f t/sec 

D. 2.4 x I0 _2 ft/sec 

E. None of these 

The writer of this problem intended that the word "small" be inter- 
preted as "negligible," Most physicists would interpret it this way . 

Many students, however, took the word at face value and correctly selected 
answer E. If such technical meanings are to be ascribed to popular words 
and phrases, they should be taught as separate objectives# 

Unless one is testing arithmetic or slide rule skills, numerical 
solutions should be made as uncomplicated as possible# Analytical solu= 
tions should be required often as both a labor saving measure and as 
practice in algebraic manipulation, but not to the exclusion of numerical 
problems . 

Finally, we mention a few additional items to watch for# In multiple 
choice problems, all the alternatives should be non-trivial and avoid any 
obvious clues (such as syntax). Problems with a remedial or enabling 
function should not express an attitude of exasperation or condescension. 
Extraneous information should not be given in a problem unless the speei= 
fie objective requires a separation of essential and extraneous information. 
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Problem Format 

The self-paced program accomodates problems In multiple choice, 
completion,, and true-false formats. The question of which format is 
most appropriate for various kinds of questions is addressed in detail 
in several references. Some rules-of-thumb regarding format which 
evolved in the physics program are presented here. 

Most problems with purely numerical solutions are presented in com- 
pletion form. Nothing Is gained by presenting such problems in multiple 
choice form, and the possibility of guessing is virtually eliminated in 
the completion format. An exception exists when a highly probable error 
can be anticipated. This error can then be incorporated as a dis tractor 
for the purpose of analysis and remediation. 

On the other hand, multiple choice is the preferred format for 
problems with symbolic solutions. The reason for this is that there is 
a great variability in an analytical solution; for example, an answer 
EF s ^ coS ® s ^ n Q 

may also be written 



Multiple choice format allows only one correct form to appear. 

Problem Review 

In addition to the obvious requirement for problem review before 
students’ use, there is a need for examination and revision after a 
field test. 

The test questions must be examined individually in relation to 
each other and to the objectives they are ■designed to test. The assump- 



K l' x = F sin (j - 0) - mg cos (~ - 0) 
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tion must be made that there is probably no such thing as a perfect test 
item, valid in every way, perfectly reliable, testing the related objective 
at exactly the proper level of complexity, with no ambiguity, and whose 
structure and content is uncorrelated with intelligence or verbal ability. 
Having made this assumption, we must then view every test item with 
suspicion. If a student answers a question correctly, we consider the 
possibility that the item was too easy, the correct answer was given away, 
or for some reason the student was able to answer the question correctly 
without having achieved the objective; if a student answers a question 
incorrectly, we consider the possibility that the item was too difficult, 
too ambiguous, the alternatives were inappropriately similar and without 
an adequate basis for discrimination within the limits of the objective, 
or that for some reason students who have achieved the objective can 
still answer the test item incorrectly. 

Obviously this approach should not cause the rejection of every 
test item as either too easy or too difficult, nor should the percentage 
of students answering the item correctly be taken as a measure of dif~ 
ficulty. Test items are never above suspicion as being major determinants 
of the answers chosen or given, and independent of achievement or non- 
achievement of the objective the item was designed to test. For example, 
if 90% of the students answer a criterion question correctly, it does 
not mean that the instruction related to that objective is highly 
effective. It could mean that, but it could also tell us nothing at all 
about the instruction or about the number of students who achieved the 
objective — it might mean only that the test item was answerable on some 
basis other than having achieved the objective. 





103 



Some subjective judgments must be made , which may not be strictly 
data based, in order to evaluate the data, find which test items provide 
the most relevant and meaningful information and make revision decisions 
for test items and instructional materials. It is unlikely that a con* 
sistent bias in favor of assuming the instruction to be better than it 
is would be present, since the entire purpose of this review is to In- 
crease course effectiveness; any inappropriate judgments of acceptability 
and adequacy would mean a wasted opportunity to improve the item. 
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PROGRESS (CRITERION) CHECKS 



We recommend that criterion tests covering all terminal objectives 
be administered at the conclusion of an average week of work (two segments 
in the physics project). 

The usefulness of tin e tests cannot be minimized. They give 
feedback to the student on the progress he is making through the course 
material, and direct his attention to those areas in which he requires 
additional work. They provide the instructor with feedback of student 
performance, upon which he should base any tutorial, lecture, or discussion 
supplementary to the basic course materials* Progress Checks also 
provide a basis for evaluation, validation, and revision of the instructional 
system , 

Although Progress Checks have been administered at the Naval Academy 
both as a course requirement and as an optional component, they have not 
been used for grading purposes. For this purpose, the Academy gave four 
quarterly Diagnostic Tests and a Final Examination. The Diagnostic Tests 
sampled all terminal objectives, but due to limitations on testing time, 
the items were primarily at the lowest levels of complexity (zero and 
one step). Moreover, the Diagnostics were administered at intervals too 
widely separated to be useful tools for effecting changes in course 

sequencing or instructional materials* 

We recommend that the Progress Checks replace the quarterly Diagnostic 
Tests as a method of establishing grades, since they serve as a better 
indication of student achievement. Progress Cheeks more closely parallel 
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the terminal objectives since they are alternate forms of the core 
problems at the same complexity level and they are administered often 
enough to provide formative feedback that should determine the next 
instruction presented to the student. Progress Checks promote criterion 
referenced evaluation--a measure of individual achievement of each ob^ 
j active- — rather than suggesting comparisons between students (norm 
referencing) , 

Both criterion- and norm-referenced tests are intended to dis- 
criminate, but they discriminate between different things. A criterion 
test discriminates between students who have achieved the objective and 
those who have not* It does not sample the behaviors described by the 
objectives* but tests them all; it is a saturated measure of the degree 
to which the student matches the descriptions contained m the objectives* 
The criterion test also provides an evaluation instrument for course 
effectiveness • 

The characteristics of a criterion test are quite different from 
those of the more traditional test* Traditional norm^ref erenced tests 
are designed to discriminate between students according to their relative 
achievement and* ideally* to produce a "normal distribution" of scores. 
Norm-referenced tests typically sample behavior and are used when students 
are to be assigned a grade or where it is necessary to identify differences 
between students rather than simply determine whether or not they have 
achieved the objectives. 

It is not surprising that many professionals regard criterion- 
referenced testing as tantamount to cheating. They have tacitly accepted 
that an examination in basic physics really should test more than was 
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taught; :it should help determine scientific aptitude 3 originality 5 and 
imagination * This may serve a useful purpose* but until we know how to 
teach these qualities it seems reasonable to separate them from examina- 
tions purporting to measure gains in knowledge or skill. 

Preparation of Progress Checks 

The content of each question was determined by reference to the 
core questions in the Problem and Solution Book* which represent the 
terminal objectives of the course. Because the core questions were 
written to test complete mastery of a terminal objective* the Progress 
Checks proved to be rather difficult questions. In our view, the 
Progress Cheek questions are "core double prime n problems. 

Questions used were of the multiple choice, true-false* or comple- 
tion type. (For the details of problem construction, see the section 
on Test Items.) Five variations of each question were produced* usually 
by rearranging numerical parameters within the text of the question 
rather than restating the question in completely different language. 
Although the Progress Checks were not used for midshipmen’s grades* 
and one Progress Check could have been given all midshipmen, additional 
copies were required for midshipmen who scored low enough to require 
retakes , 

Time alloted for midshipmen to complete a Progress Check varied 
with instructor, but was usually 50 minutes* Many midshipmen frequently 
took longer , Upon completion of the Progress Check, the instructor* 
having been provided an answer key* graded the Progress Check and either 
returned it to the midshipman for review or passed it to clerical person- 
nel for statistical analysis and preparation of retake Progress Checks. 




We think that the student should receive the graded test immediate] y 



whether or not he successfully completed it. This feedback to the student 
is a dynamic element of the course. 
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SAMPLE PAGE 



PROGRESS CHECK 



T , 0 


25 




RR 


The 


principle of the conservation of energy 


may be written as 


(A) 


Energy may be 
from one kind 


created or destroyed and 
to another. 


transformed 


(B) 


Energy may be 
but it cannot 


transformed from one kind 
be created or destroyed. 


to another, 


(C) 


Energy may be 
kind to ar,oth<5 


created and transformed from one 
‘r, but it cannot be destroyed. 


(D) 


Energy may be created or destroyed, but 
transformed from one kind to another. 


not 


T.O. 


26 




RR 


The potential energy for an object of mass m 
above the surface of the Earth is 


a height h 


(A) 


1/2 mb 2 






(B) 


1/2 mgh 






(C) 


mgh 






(D) 


2 mgh 






r,o. 


27 




■ t r 



The bob of a simple pendulum (mass m) is displaced from its 
equilibrium position such that the string (length R) holding 



side 


, how 


(A) 


R/3 


(B) 


R/2 


(C) 


2R/3 


(D) 


R 



3 
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PRETESTS 



In order to have a comparison measure of how much learning progress 
a student is making, Pretests are generally given at the start of a new 
unit of work, But, more important, a pretest should also serve as an 
indicator to shunt a successful student onto the next major topic area, 
without requiring him to spend precious time with unnecessary Instruction 

The final version of the physics course incorporated both functions 
of pretesting by providing a core problem for each topic. If the student 
is successful on this core problem, he can bypass the subordinate mate- 
rial. An Incorrect response on the presented core problem indicates 
that the student is In need of additional instruction on the objective; 
his path through the enabling problems can often document his particular 
difficulty, and obviate it. 

Originally pretests in the physics course were administered everv 
four weeks, A test covering so large a portion of course objectives 
could only sample the Instructional material, provide skimpy data on 
incremental improvement from pretest to posttest, and could not serve 
simultaneously as a basis for branching students ahead on those topics 
already achieved, 

To remedy these drawbacks, pretests were given weel iy during the 
experimental run of the program. This procedure, however, also had 
attendant faults. Because time for testing ana study was at a premium 
at the Naval Academy, weekly administration of pretests made an already 
demanding schedule an unrealistic hardship. Discussion with students 



and faculty members enlightened the project staff to unspoken sanctioning 
of students' not expending much effort on attempting pretest problems 
while the data merely indicated inability of the students to achieve the 
problems. Weekly pretests, while not covering as many topics as the 
previous method, still were inadequate for providing indication for 
forward branching. Our solution of "pretesting" by core problem at 
the objective level removed these deficiencies. 
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DIAGNOSTIC TESTS 



Diagnostic tests were prepared for the purposes of providing review 
for the students and as input to the administration for assigning grades. 

We feel these a m can be met more satisfactorily through review segments 
in the Problem and Solution Book and grading based on achievement as 
indicated by progress checks. We therefo e recommend the elimination of 
diagnostic tests. 

We propose that a student who ha- passed all the progress checks 
(even when retests were needed) be assigned a Tnvnwuvr, gTQ,d@ of C (or 754') • 

In this way the student is tangibly rewarded each week for his progress 
by accruing a substantial fraction of his final ft rade. The differential 
between grades C and B can be established by the mid-term and final} and 
between B and A by tests on the enrichment material. 

At the Naval Academy, where continual testing for the purpose of 
assigning grades was required, diagnostics were prepared for each quarter 
of the course, This constant testing, we believe, put an additional strain 
on the already overburdened physics students. The tests were administered 
at intervals too widely spaced to be useful tools f • r effecting changes in 
course sequencing or instructional materials. The scope of each test was 
so large thct the sbjectives could only be sampled, and those items included 
could only be tested at the lowest complexify levels to keep the test length 
down to the allotted time. We think, therefore, that basing grades on such 
unrepresentative test items presents an unreliable picture of student 
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ach ievement * 



A sample of diagnostic test questions is attached. Usually five 



paic^’el questions for each terminal objective vtre prepared so that 
several very similar tests could be constructed for different student 



sections. This procedure prevents transfer of test information to stu- 
dents who will be taking the test at a later hour. 
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SAMPLE PAGE 



DIAGNOSTIC TEST 



T.O. 28 



CU 



The center of mass of a straight drinking straw 

(A) is the axis of the straw, 

(B) is the entire outer surface of the straw, 

(C) is the point on the axis of the straw equidistant from 
either end of the straw, 

(D) is the ring of points around the center of the straw, 

T.O . 29 CU 

When a group of particles is in motion, the center of mass 
moves as though it was a particle with a mass equal to the 
sum of all the masses of the particles in the group. 

What is the force exerted on this fictitious particle? 



(A) 


the average of 


all internal forces in the particles 


(B) 


the sum of all 


internal 


forces in the particles 


CC) 


the average of 


all external forces on the particles 


(D) 


the sum of all 


external 


forces on the ^articles 


'H 

O' 


30 






Two 
with 
of 4 

the 


bodies each of mass 3 kg are moving eastward; one 
a velocity of 2 m/sec, the other with a velocity 
m/sec. The magnitude of the total momentum of 

system is 


(A) 


6 kg-m/sec 






(B) 


12 kg^m/sec 






(C) 


18 k g-m/see 






(D) 


60 kg-m/see 
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FINAL EXAMINATION 



The physics final examinations at the Naval Academy typically comprise 
55 or 6Q multiple choice questions to be completed in three hours. They 
are constructed by Academy professors to be norm-referenced tests (see the 
discussion on norm-referencing and criterion-referencing in the Progress 
Checks section) intended to assign a spectrum of grades. Both self-paced 
and "conventional " student sections took the same final examination ad, 
although the self-paced group performed at least as well as the conven- 
tional. group , the physics course developers felt that this kind of test 
does not measure the objectives which were taught by the program, (For 
a full discussion of the Fall 1970 examination , see the report on Course 
Eva ] -ati ~n * ) 

limit ms on . final examination allow only a sampling of 
the objectives. time constraint seems also to turn these examinations 

into speed tests with about three minutes all oted per problem. Contrast 
this demand for rapid response with the approach fostered throughout the 
program. Self-pacing encouraged methodical and contemplative work to 
synthesize all the elements of complex problem solving* Unless the avail- 8 
able time for completing final exams is expanded considerably, these tests 
cannot be regarded as an acc a measure of criteria attainment in the 
sell-paced program. 

It is important to recognize that high achievement on rapid answer 
tests is exactly what many institutions do want as their basic course 
objective. A great deal of pressure is applied in this direction by 
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license examinations, and others. Course deveLopers must realize rhqr 



to he properly trained for the race. Thus, appropriate time limits 
should ha set on each problem, and core problems should he sufficiently 
uncomplicated to be executed in the required time. 

Having said this, we want t n express our objection as educators to 
any system which makes test demands so unrelated to the actual use of 
knowledge. A professional physicist or engineer does not spend his time 
working rapid fire at little puzzles; he is required to solve many com- 
plex problems. The course objectives should prepare students for real 
activities, not for unrelated test skills. 

We th'nk that a final examination should be given. It is a good 
educational tool in that it prompts students to review and synthesize 
the course work. Since time will not allow more .ban a sampling of 
objectives, the final exam should count only as a fract-,»n of th_ grade 
(probably less than 501). In preparing the exam we suggest the procedure 
outlined in the sections on Test Items and Progress Checks. 
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PROCESSES 
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COSTS 



The cost ef f ectiveness of an educational system is often ^aken as 
the number of criteria attained per unit cost. While this measure has 
a certain appeal, it is an abstraction insofar as existing educational 
instil tions do not graduate students on the basis of the total attained 
criteria . Thus, a more realistic measure of cos h effectiveness is simply 
the number of students processed through the course (regardless of achieve- 
ment) per unit cost 

Instructor salaries constitute a major cost in education. Academy 
policy limits class sections to no more than twenty students per instruc- 
tor. Although many more students can be served by one instructor using the 
self-paced program, the Academy constraint prevents the realization of a 
gain in coat effectiveness over the conventional system. Generally, cost 
effectiveness increa. <>s with increased numbers of students. In some cases 
a gain of 100% or more in cost effectiveness seems likely with the use of 
the existing program. 

Another major cost of education is overhead. Through the use of the 
self-paced program, there exists the possibility of reducing the required 
classroom space -a factor which would further enhance the program cost 
effectiveness . 

For educators who are considering developing their own self-paced 
program, che attractive possibility of a highly cost effective program 
must be amortized against development and installation costs. These costs 
vary greatly, depending upon the materials and sophistication desired. 
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Development Costs 



Wages have a wide range of variation depending upon geography * the 
nature re the institution, and personnel qualifications nut associated 
with program development. Man-hours are a more stable measure of costs 
for course writers and developers- The estimites which follow are based 
upon minimum man-hours required by experienced personnel for each function 
These figures should be increased for an inexperienced team by about 
thirty to fifty percent to estimate an upper bound on wage costs: 



Selecting course content 



Days per Semester Course 

10 



Wr^ing Course Objectives (core statements) 
1-1/2 days /segment x 30 segments /semester 

Writing Information Panel s 

3 days/segment x 30 segments /semes ter , * , ♦ 

Wrj ing core problem solutions 

2 days/segment x 30 segments /semester . * . . 

Writing enabling problems and solutions 

3 days/segment x 30 segments/semester , * * 



Writing core prime problems and solutions 
1 day /segment x 30 segments /semes ter 

Preparation of Study Guide 

1/2 day/segment x 30 segments/semester 



M 



• * • 



90 



i * * * i • o0 



90 



30 



15 



material . . . . 



f 9 • 



* 0 



1-1/2 days/segment x 30 segments /semester 45 



Preparing remedial sheets 

1/2 day/segment x 30 segments /semester . . 15 

Classifying learning objectives 

This function is included in the estimates 

for writing core and enabling objectives 0 



119 



125 



Days per Semest e r Course 

St aboard development 

Approx, 1 storyfcoard/segment x 1 day/storyboard 

x 30 sonants /semester 30 

Writing illustrated text 

1-1/2 days/illustrated text x approx, 30 

illustrated texts /semes ter 45 

Developing talking book 

Alter illustrated text into verbal narrative 
and tape voice over 

1/3 day/talking book x 30 talking books /semes ter 10 

Consulting for videotape presentatio n 

1 day/videotape x 30 videotapes/semester , 30 

Technical editing for all written materials 

2 days/segment x 30 segments /semes ter 60 



Those institutions which will undertake the development of their 
own videotapes or films will need a Media Director, Producing thirty 
good quality films or tapes of fifteen or twenty minutes 1 duration will 
take ten to thirty weeks depending upon the sophistication being sought — 
fifteen weeks is probably realistic for acceptable quality videotapes. 
Shooting occupies only a fraction of the time to prepare the audiovisual. 
Two hour sessions were quite sufficient for the taping of a twenty 
minute videotape. Camera crew costs (2 to 4 cameramen per session) may 
be computed on this basis. 

The physics program employed both a non- technical editor and a 
coordinator (see Production) , Both these functions can be met by one 
individual if procedures and formats have been established as presented 
in this document. The coordinator must be available full time for the 
duration of the development (not less than twenty weeks for a one-semester 
course). The coordinator also supervises the typists who must type all the 
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software. An attempt was made to type a finished mechanical at this 
point. Xerox copies were furnished to proofreaders and subject matter 
experts } and their corrections and changes were entered on the mechanical 
by using paste-ups. 

Each segment was approximately thirty-five pages of single spaced 
elite type so that over 1000 pages were typed (35 pages x 30 segments) for 
each semester. This technical typing is much slower than routine typing 
of unfamiliar jargon because of the equations requiring Greek symbols, 
subscripts and superscripts, built-up fractions, etc, A reasonable esti- 
mate for 1000 pages is almost three months of technical typing time for 
production of finished mechanicals for offset printing, 

A considerable number of diagrams and drawings had to be prepared 
for the physics project, and we suspect that other sciences will require 
a like number. Our full time artist was able to do all the artwork 
associated with one segment, including that required for audiovisuals, 
in an average of five days. This production rate was only possible, how- 
ever, due to uncommon dilligence; we think a more reasonable estimate 
is about seven days per segment or forty-two artist-weeks per semester 
of material* 

Costs for film or videotape development varv greatly depending upon 
the production and whether or not :he equipment is owned or leased, A 
prevalent rule-of-thumb for producing film (and videotape as well) is 
$2000. par minute of film on rented equipment. Acceptable quality film 
or tape can probably be made for $500* per minute or less, but budgets 
should not be planned for less without expert advice. 

Many institutions have their own printing facilities and can realize 
a savings by reproducing software in-house. Local rates for commercial 



ERIC 




121 



printers in the Long Island area varied from $4,50 to $5.00 per page. 
Quantities had little effect on the price# Wet- to-reveal sheets for 
Study Guides cost us approximately 9if per sheet in quantities of 1000, 
and the physics project averaged two of these sheets per segment. 

Operating Costs 

Academy processors have estimated that in a classroom of self -pace 1 
students they can serve at least twice as many students as in a conven- 
tional elasoroonu The on-site professor spends all of his time in answering 
individual questions# Some students appear to need the attention of an 
instructor, not because he is the only source of the information being 
sought, but to satisfy some subjective dependence# 

A self-paced course requires considerably more grading and processing 
than a conventional course. In the physics program, instructors were 
originally assisted by a clerk and computer grading and analysis# 

A clerk is an important asset during the first trial run because minor 
difficulties generated by oversights, errors, and miscalculations are 
much more easily corrected when instructors have assistance. Our course 
evaluators are now of the opinion that clerks (beyond the trial run) and 
computer assistance are desirable but not anv more necessary for self- 
paced classes than for conventional sections. The increased load in 
grading and evaluation is probably matched by a decrease in preparation. 

It is evident that a good many trivial questions arise in the self- 
paced program. Perhaps more than half of all questions can be readily 
answered by personnel with considerably fewer qualifications than the 
regular instructor. Senior students with the appropriate major or graduate 
assistants can serve well in this capacity, but a regular instructor must 
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be present to treat the more difficult questions, "’his situation can be 
realized when large numbers of students (40 or more) sit in one room or 
auditorium so that the n parapro£ essional" personnel can move among them, 
referring the more time-consuming questions to a professor in the same room. 
Of course, we could not institute this arrangement at the Academy, so we 
do net know what proportion of paraprof essionals to instructors is reasonable. 
Classroom space and its attendant overhead costs can also be reduced 
by the use of a self-paced course. Independent study does not need to be 
conducted in a classroom except for viewing videotapes or films* These 
can be provided in a separate audiovisual room which services many such 
programs# The on— site instructor does not need a classroom; he may be 
made available through office hours scheduled for that purpose. Again, 
it was not possible to experiment with such arrangements at the Naval 
Academy so we can only speculate that the program would be successful 
even without class meetings. 
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PRODUCTION 



Production of materials for student use underwent extensive improvement 
with each modification of the project. Format changes suggested several 
of the revised methods , while experience in using the materials suggested 
others , 

All materials from the course writers were edited for clarity, con- 
sistency s grammar, and style by a production editor. The edited material 
was then checked by subject matter experts to ensure that no changes 
interfered with technical accuracy* 

At the same time our technical artist prepared the required artwork* 

This included diagrams, graphs and symbols for all the written material, 
and for several of -he parallel path media (illustrated text and talking 
book). The other media were also prepared at this time. Videotapes, using 
storyboard sketches which parallel other media, were taped both at the 
Academy and at NYIT, taking advantage of the talent on both campuses. 

Manuscript for reproduction was prepared using IBM Sslectric typewriters 
expanding the usual keyboard to include Greek and most mathematical symbols. 
Unusually large summation signs, etc., were added to the manuscript by 
pressure sensitive lettering. The manuscript then underwent several reviews: 
(a) manuscript was proofed against the original copy; (b) a coordinated 
review was made, including placement and correctness of artwork, and flow 
through the problems and solutions as compared with the study guide for the 
segment, pagination, etc.; (e) a technical review was made for subtle errata 
which might have been overlooked by the nontechnical editor. The finished 
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mechanicals were then given to the printer for offset reproduction, punching, 
and binding. 

Our experience with early phases revealed that mimeograph was a poor 
reproduction process. Hand^inserted symbols and all diagrams are difficult 
to put into the mimeograph stencil. Moreover, the legibility is poor, the 
contrast is inconsistent, and typing errors are difficult to correct. We 
found that Xerox photocopy was much more satisfactory for limited quantities. 
For larger quantities, the offset process was most convenient and inexpensive 

The coordination of production must be centralized with all production 
progress and requirements carefully tracked since so many diverse activities 
proceed at zh& same time. Writers, artists, typists, editors, printers, 
all work on different stages of material preparation. Thus, records must 
be kept up to date on all aspects of this effort. A great many record 
keeping devices were used and replaced in the evolution of the program. 

The most recent and useful charts include a Chart for Material Location, 
Segment Process Chart, and a Study Guide Process Chart (see the samples 
in this section; . 
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CHART FOR MATERIAL LOCATION 
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STUDY GUIDE PROCESS CHART 
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STUDENT ORIENTATION 



Because the self-paced course does not follow traditional classroom 
operation* there is a need to acquaint the student with course procedures. 
A description of the course, its structure and materials are detailed in 

the Student Manual which is prepared for student orientation. Our manual 

i 

will serve as a model for other self-paced courses . 
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STUDENT MANUAL 

1, DESCRIPTION OF THE COURSE 

The self-paced physics course differs from conventional courses in 
a number of ways. It is largely student-managed programmed instruction. 
Most of your learning will be derived from reading carefully selected 
passages in excellent textbooks , simplified written discussions of the 
highlights of the various subject areas , and the use of audiovisual 
aids in the form of videotapes, "talking books'’ * and brief, meaty illus- 
trated pamphlets called Illustrated Texts. An instructor will be avail- 
able for tutorial assistance as well as diagnosis of your prograss. 

The format of the course permits you to monitor your performance 
and achievement by means of instant feedback from the visual response 
mechanism to be described later. 

In addition to self-paced theoretical instruction, you will also 
spend an adequate amount of time in the physics laboratory and attend 
a demonstration-lecture periodically. 

You will always know in advance when a check quiz or an evaluation 
test is to be given. As a matter of fact, you will determine for your- 
self when progress checks will be administered to you. In addition to 
other periodic tests, a final examination will be used to evaluate your 
achievement . 



2. COURSE STRUCTURE 

Assigned reading — From standard textbooks, coded as follows: 

HR means Halliday and Resnick, PHYSICS FOR STUDENTS OF 
SCIENCE AND ENGINEERING, fifth edition, combined form; 

SZ means Sears and Zemansky, UNIVERSITY PHYSICS, third 
edition , complete; 

AB means Albert Baez, THE NEW COLLEGE PHYSICS - A SPIRAL 
APPROACH, first printing; 

3W means Shertley and Williams, ELEMENTS OF PHYSICS, fourth 

edition. 

The required or prime reading assignment for each segment of the 
course will be identified by one or more asterisks before the chapter 
numbers. The remaining reading is to be considered supplementary. A 
typical reading assignment and its interpretation will be presented as 
a sample later in this Manual. 
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STUDENT MANUAL 



For maximum ef f ectiveness * all assigned reading should be completed 
before you begin work on the programmed instruction. This first read- 
ing need not be exhaustive because it is anticipated that you will return 
to certain sections of it time and time again as you work through the 
segment - 



Information Panels— ksi&e from your textbook reading, much of 
your factual and procedural information will come from Information Fanels 
presented in the PROBLEM AND SOLUTION booklet for each segment. These 
Panels are concise discussions relating to the principles and methods of 
solution involved in the accompanying problems. If you should find that 
you do not fully understand the material In the Panel for a given section 
of your work, you would be expected to return to the textbook assignment 
for clarification , 

Audiovisuals — These are important adjuncts to your reading and 
problem solving. When you are directed to work with a specified audio* 
visual, you will usually be given the option of selecting one of three 
media of presentation. 

Videotape: a demonstration accompanied by a discussion that 

you view on the screen of a small videotape playback; 

Talking Book: a set of carefully constructed pictures and 

diagrams accompanied by an audiotape lecture; 

Illustrated Text: a set of pictures similar to those used for 

the Talking Book accompanied by a formal written discussion matched page 
by page to the illustrations. 

1 no gross Cheeks-- groups of relevant questions which you must answer 
after completing a specified number of segments. These checks will be 
used for diagnosis, progress evaluation, and tutorial assistance should 
the latter be needed. 

Quarterly Diagnostics — special test forms administered periodically 
to assist your instructor in diagnosing possible weak areas in your 
learning pattern, and to enable him to prescribe remedial work where 
required. The Quarterly Diagnostics will also be used to evaluate your 
achievement . 

Pinal Examination— standard examination which provides information 
relative to your final grade. 

Enrichment Packages-- for those students whose progress 
additional, higher level material; to be a student option. 
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3. PRINTED LEARNING MATERIALS 

PROBLEM AND SOLUTION BOOK* (Hereafter referred to as the P&S.) This 
is bound stud}? material containing the work for three or more segments in 
a volume. The entire course consists of 45 segments for the semester. 

The P&S material in a given volume will contain blue title sheets between 
segments to help you find the one you want quickly. Each P&S contains? 

(a) A problem section in which the questions and numerical prob= 
lems are presented in strict numerical order, to be worked on in se- 
quence * 

(b) A solution section in which the correct methods of answering 
questions and solving problems are presented in scrambled order. Many 
of these solutions are terminated by additional "true-false" questions 
to be answered immediately after you study the individual solutions, 

(c) Information Panels strategically interspersed throughout the 
problem section, 

STUDY GUIDE, This is just what its name implies: a written guide 

that you must follow step-by-step, strictly in the order presented, to 
work your way through the problems, information panels, audiovisuals, 
reading, solutions, and other check points. The remainder of this Man- 
ual will be devoted to an explanation of the way in which all these 
aspects of your learning are related, 

4, HOW TO USE THE STUDY GUIDE 

Please refer to the sample Study Guide which is the last page of 
this booklet. It is a partial mock-up of a segment that doesn't really 
exist, and will be used for explanation purposes only. If you are to 
understand how the system works, if you are to avoid blunders when you 
start work on your first actual segment, you must walk through the fol- 
lowing explanation without missing a step. Take your time; be abso- 
lutely certain you understand each maneuver perfectly. If you need help 
in interpretation, ask for it. 

Before you begin work on any segment, ascertain that you have the 
correct STUDY GUIDE by checking the number near the upper right-hand 
corner, then complete the heading on each STUDY GUIDE sheet. 

Another preliminary step: look at the bottom of the STUDY GUIDE 

sheet and note the number of pages you should have in your hand. Pew 
STUDY GUIDES contain more than two pages. Be sure you have what you 
need before you start work. 
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The letter P above the left column taeans "Problem Number ; " the 
STEPS are also numbered to Indicate the sequence of things you must do 
other than problem solving. 

All right* Let's go through the sample. 

Step 0,1 The reading assignment for the segment. The required 
reading is in Halliday and Resnick, paragraphs 49-3 
through 49-6 and paragraph 49=9 . The slash-bar (/) 
always means from one paragraph through the other, in- 
clusive , The supplementary reading is in Sears and 
Zemansky, paragraphs 45-6, 45-7, and 45-11, This read- 
ing should be gone through at least once before continue 
ing. 



Step 0,2 When you have finished your reading, turn to the first 
page in the P&S for this segment. Read the Information 
Panel, be sure you understand if fully, then continue, 

1 This is the first problem in the P&S, Note the overscore 

and underscore lines. These indicate that the problem 
is a core type, required of all students in the course 
You will find this problem boxed for the same reason in 
the P&S, The problem you find in the P&S as number 1 
is : 

How many gallons of regular gasoline could you have 
purchased with 5 Martian zilches in Septimus, Ohio 
in the year 1960 and still have some change left over? 

A. 1 

B. 2 

C, 3 

D, 4 



Now obviously, to solve this problem you would have to 
know the price of gasoline per gallon in U.S . currency 
and also the equivalent buying power of a Martian zilch. 
Presumably, your reading and the Information Panel con- 
tains this information but let us suppose that you 
didn't do any of the reading and so didn’t know the 
answer, So--you 1 re about to make a wild guess, let's 
say, answer A. At this point you rub the "reveal" 
pen provided all over the inside of box A for the 
first question. As you do so, you will sea an X appear, 
showing that the selection was incorrect. Do it now; 
reveal the X in box A with your pen. (Best results are 
obtained by rubbing the pen lightly over the surface, 
then waiting a few moments for the revealed information 
to darken,) 
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Making another stab at it, you choose answer B and use 
the r^n bringing out another X. Trying C, you find that 
the pen reveals the characters 29 [a]. This tells you 
to turn to page 29, item [a] in the P&S where you will 
find the full explanation of the method used to solve 
the problem. For this core question, you will also 
find a very short true-false qu.CS tion immediately after 
the correct solution. This question reads as follows: 

A Martian zilch is the equivalent of three U,S, 
nickels. True or False? 

You must now use the reveal pen on either the T^box 
or the F-box for question 1. 

If you make the correcL true^false selection, a V^will 
appear in the box. If you choose incorrectly, an X will 
appear in the box. The true- false questions are usually 
so simple that you will be permitted few, if any, errors 
in this part of the work. Getting one of these !~F f s 
wrong is a pretty sure indication that you are not read- 
ing the solutions* You must avoid this* 

Let's go down to the next step. 

Step 1,1 You are now being given an option. If your first choice 
was correct, you will be permitted to skip over the next 
four questions and advance to the next Information Panel, 
If you answered incorrectly, even once, you must go 
through the remedial loop consisting of questions 2 
through 5, 

Ue are assuming that you mis: ed question 1^, so let's 
go through this loop together, 

2 Problem 2 in the P&S . It is not scored, hence it is 

not a core problem. It reads as follows: 

It is predicted that a gallon of regular gasoline 
will sell for $1,05 by the year 1998, If this is 
roughly 3-1/2 times the price of gasoline in I960, 
how much did one gallon cost in I960? 

This is not multiple-choice , It T s a completion type of 
question where you must write in the answer* So, write 
your answer on the line below the rectangle for question 
2, The answer is, of course, 30c because $1,05 is 3-1/2 
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times 30$. After writing it in, reveal the answer in 
the rectangle with the pen; the answer 30? will appear 
accompanied by the referral page and item 14[c]. Turn- 
ing to the referral, you find the solution worked out 
for you to check your own thinking. Problems that are 
not core types are not accompanied by true— false check 
questions, so you're ready to go to question 3. 

Let's interrupt the sequence for a moment. Even if you 
were able to answer the original core question correctly 
the first time, you should go through the remedial loop 
anyway if you have any doubt at all about the method of 
solution or the answer. You may have guessed at the 
right answer, or you may have made two errors that can- 
celed out. In any case, if you feel that your choice 
of the right answer was a fluke in any way, we urge you 
to go through the remedial loop. 

Problem 3 in the P&S ; it is not a core problem. Here 
it is : 

Ten Martian zilehes will buy exactly the same number 
of 2-1/2 inch McIntosh apples in a given market on a 
given day as two U,S, dollars. Thus, one zilch is the 
equivalent of 

A. 10? 

B. 20? 

C. 40? 

D. 60? 

A glance at the STUDY GUIDE corroborates the fact that 
this is another multiple-choice question. Apparently 
10 zilehes is the, equivalent of $2.00, so one zilch 
must be worth 20?. This is answer B, so if you use the 
reveal pen in box B you will bring out the Instruction 
18 [b] indicating that page 18, item [b] in the P&S has 
the solution. Whether you were right or wrong in your 
selections, it is important that you read and understand 
the solution. If you had chosen any answer other than 
B, you would have revealed an X as before. There is no 
true-false question, hence you can now go on to question 
4. 

Here is your first modified true-false question: 

True or false? Five martian zilehes will purchase 
more milk than 20 U.S. dimes. 
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Note the italicize I word. Read the statement and (a) 
if you decide it is true , simply ink the T-box on 
the STUDY GUIDE; (b) if you feel that it is false, 
write a word that can replace more and thereby make 
the statement true. After you have written the correc- 
tion word on the line under the F rectangle, then, and 
only then, you are to reveal the answer with the pen* 

In this particular instance, the correct answer is 
"false" and you would write in the word "less in place 
of more , Your reveal pen will bring this out* too* 

If you had selected "true" as your answer, the pen 
would have revealed an X inside the T-box. So, after 
writing "lees" you would see revealed.' "less (21 [d] ) . " 
At this point in an actual lesson, you would turn to 
this page and item in the P&S and read it carefully 
before continuing the sequence. 

Continuing with the emedial loop: 

5 Another multiple-choice question: 

In order to have filled your 18-gallon tank with 
gasoline in 1960 in Septimus, Ohio, you would have 
spent at least 

A, 15 zilches 

B, 21 zilches 

C* 23 zilches 

D, 27 zilches 

The correct answer is, of course, 27 zilches since each 

zilch is worth 20c and each gallon costs 30c * so you 
would reveal box D and find inside the instruction 
"27 [b]," After reading the solution, you again encounter 
a check T-F question which is then answered as before by 
revealing either the T or F box in question 5, Any 
answer other than D above would have revealed an X just 
as described for the previous multiple-choice question. 

Step 5,1 Everyone is now expected to devote some time to the 
Information Panel, "The Currency of Venus" and then 

Step 5*2 select the medium he wants for running through the audio- 
visual, COINAGE AND BILLS OF THE INNER PLANETS * 

After that is finished, everyone starts once again on an 
equal footing with the core question 6, 




137 

143 



And so forth. 



U S. N/VAL ACADEMY 



STUDY GUIDE 



SELF-PACED PHYSICS 



p 


STEP 


NAME 


P 


STEP 


SECTION SEGMENT 60 


i 


0.1 

0.2 

1.1 


Reading: HR *49-3/49-6; *49-9 

SZ 45-6, 45-7; 45-11 

Information Panel, "The Currency . 
of Mars" 

A B C D T F 

□□□□ □□ 

If your first choice was correct, 
advance to 5,1; if not, continue 
sequence * 






, ’ - - - ’ - - 

Note* In this sample n walk 
through", we have not in- 
cluded the Information 
Panels nor any sat-up P & S, 
The problems that would nor- 
mally appear in the P & S 
are given in the Student 
Manual for explanation pur- 
poses , 


2 


















(ans) 






AND SO FORTH 


3 

4 




A B C D 

pnnn 










m i 












(ans) 








5 




Suu 


D 


T F 

nn 

p ■ 1 t ' i ■! 










5.1 


Information Panel 
of Venus 11 


, "Th 


e Currency 










5.2 


Audiovisual, COINAGE A] 
OF THE INNER PLANETS 


MD BILLS 

T f 








6 






nn 












(ans) 










6.1 


If your answer was correct, ad- 
vance to 9.1; if not, continue 
with sequence. 








7 






















(ans) 








0 













ERIC 



144 



FACULTY ORIENTATION 



During the early stages of development , faculty orientation was 
primarily through Informal progress reports at regular weekly meetings 
of the Academy's Physics Department, These informal reports continued 
although the general faculty was not involved in actually administering 
the experimental course during the early phases of the project. 

The faculty was encouraged to visit the multimedia room* review the 
course material and discuss the project. Their impressions and comments 
were recorded ? and their recommendations and questionnaire responses were 
Incorporated into the revision process. 

The faculty review was of great assistance to the course developers 
in the revision effort, but in addition it provided the faculty with 
"hands on" familiarity with the Instructional methods and materials, which 
in turn increased their appreciation of the project. 

Since the degree of cooperation and usefulness of faculty members 
was very closely correlated with their degree of involvement in the develop- 
ment of the program, course evaluators strongly suggest that the on-site 
instructors participate in the development effort. Naturally, this is not 
possible when installing an existing program, but in such cases instructor 
participation is not as critical, since a revised and validated program is 
nearly self-standing. 

The Instructor Manual is reproduced here as a self-explanatory descrip- 
tion of the tasks assigned to the on-site instructors. 
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PREFACE 



This? manual was prepared as a reference and guide for Instructors 
of the Naval Academy Self-Paced Physics Course. Additional orientation 
is provided by the Course Manager. 

Contained herein are* 

1. Notes to the Instructor, 

2. A description of the Management Sequence, and 

3. A flow-chart which reflects a general overview of 
the operational functions of the course. 

It is suggested that the Instructor familiarize himself with the course 
materials and the following student "hand-outs"- 

Course Policy 

The Student Manual 

The Self-Paced Laboratory 
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NOTES TO THE INSTRUCTOR OF 
SELF-PACED PHYSICS 



1 . Introduction 

The methods and operation of the self-paced physics course may seem 
strange to new instructors as well as to the students. This information 
is presented to assist the instructor in developing his individual class 
policies. It is presumed you are familiar with the Student Manual and 
Course Policy Statement, 



2, Objective 

The objective of the couiie is to enable each midshipman to complete the 
tasks defined by the Terminal Objectives (TOs), If you have not done so 
previously, you should read the TOs, as they constitute the most accurate 
definition of course content. Because of the way the Problem /Solution 
books have been constructed, successful completion of all the core question 
ghoul 1 cover all the TOs, Since the core questions were also designed ro 
provide a path for fast students, they are frequently complex problems 
that combine elements of several TOs. Due to the limited time available 
for testing, the body of TOs is sampled randomly during Progress Checks 
and Diagnostic Tests, 
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NOTES TO THE INSTRUCTOR OF SELF- PACED PHYSICS ( Coni' d) 



3, Class Atmosphere 



There are few constraints on how you use class time to move the 
students through the material. If your class size permits, you are en- 
couraged to use Room 203 as your regular classroom. Initially, a 
certain amount of encouragement may be needed to steer the midshipmen 
to the various media. You should try as many of the media as time 
permits yourself so you can recommend a particular Audiovisual if a 



midshipman is having trouble in a specific area. You may wish to add 



additional demonstrations or conduct smal 1 topical lectures occasionally. 
Comprehensive reviews prior to Diagnostic Tests are frequently given. 



3. Student Progress 

One of the by products of the course organization is the early identifica- 
tion of potential failures, before they reach the Diagnostic Checks. This 
early identification can be done most effectively by careful screening of 
study guide responses and progress check responses. The individual 
prescription for assistance is in your hands, but the early identification 
of these individuals and the variety of materials available should provide 
you with considerable flexibility. 
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NOTES TO THE INSTRUCTOR OF SELF-PACED PHYSICS (Cant'd) 



5, Areas of Concern 

a. Minim inn Lecture, You, as well as some of your midship- 
men, may feel uncomfortable, initially, because you are not conducting 
lectures during most of the class time. Experience has shown that most 
students adapt readily to the self-paced class routine within four to six 
weeks. You may choose to lecture frequently; however, you will probably 
have little time left to grade progress checks or counsel slow students, 
except in El (Extra-Instruction) Sessions, Another by-product of the 
course organization is to move a substantial amount of student counsel- 
ing and remedial work into the classroom. 

h. Student Prog ress, Because of the groat amount of material 
covered by the course, you will soon find students dropping well behind 
the average (or, from your view, a desirable) class progress. Your 
success in keeping the class moving will be limited only by your 
imagination. One reason for the apparently slow class progress may be 
confusion between a very weak physics student and a good student who 
chooses to ‘‘pace 11 himself to the speed of slower classmates. Careful 
screening of study guide and progress check responses can usually 
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separate the two. 



The Management Sequence 



1, Each student is issued one prime textbook; at least two other 
supplementary text? are at all times available in physics or in the 
library. 

2, Each student is issued a Student Manual intended to supply 
the student with all the procedural information required. 

3, Course work begins with the issuance of Segment 1 of 
Problems and Solutions and the Study Guide for the same Segment. The 
Study Guide is a latent image type on which sequencing information is 
revealed by means of a special crayon, 

4, The Study Guide features are; 

(a) A reading assignment indicating prime reading and 
supplementary reading, both clearly identified, 

(b) Core problems identified by score lines over and 
• under the problem number. 

(c) Remedial loop problems ("enabling problems"). 

The instructions for short-circuiting the loops, or following them, are 
contained in the Study Guide for each individual set. 

(d) Titles and directions for Information Panels con- 
tained in the Problems and Solutions. 
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(e ) Titles and directions for Audiovisuals* These are 
available in three formats: 

{ 1 ) Video tapes; 

(2) Talking Books; 

(3) Illustrated Texts 



(f) Homework assignment* generally in the form of 



additional problems in the prime text* 



5. The Problems and Solutions features are: 

(a) Section 1: Problems and diagrams in numerical 



sequence * 



(b) Core problems identified by enclosing each one in a 



box* 



(c) Information Panels preceding core groups* 

(d) Scrambled problem solutions: directions for reaching 
solution is revealed only in the Study Guide when correct answer is chosen* 

(e) Each solution for core and core-primed questions is 
followed by a true- false question whose answer is derivable from the solution 
to which it pertains* These TF's are answered in special boxed sections of 
the Study Guide, NOTE* Each core problem which is answered incorrect- 
ly requires that die student follow die remedial or enabling loop which 
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always concludes with another problem having the same conceptual 
basis as the core problem initially missed. Such problems are called 
"core-primed. " 

(f) The scrambling process used for the solutions is 
extreme] y difficult to compromise. The time required to short-circuit 
the response pattern is expected to be too great to make it worthwhile. 

6. The Progress Check. This is a form of test which follows 
a unit of work, usually three successive Segments. The Progress Check 
is graded by the teacher. The performance of the student is evaluated 
and he is then guided into one of the channels indicated below. To be 
eligible! for the Progress Check, the student must submit to his instructor 
all of the relevant revealed Study Guides for that unit. 

(a) Using a predetermined cut-off grade, the student is 
given the "go" signal if his performance is above this level. He is also 
given a set of remedial suggestions in the form of reading, programmed 
material, films, etc. 

(b) If his performance falls below the cut-off, he is given 
a "stop" signal with remedials, after which he re-takes a Progress Check 
Questions on these checks will be randomized so that no two students ever 
take exactly the same examination, nor does the same student take the 
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same check on the second round. 



(c) If his performance falls below cut-off on'the retake, 
he will be given individual tutorial assistance and required to take a 
third test. Disposition of the student after the third failure will be left 
to the chairman of the physics committee at the Academy. 

7, Quarterly Diagnostic Tests, These tests will be carefully 
generated to test for recognition and recall, understanding of concept, 
ability recognize concepts which appear in problems, and ability to 
solve problems. These tests will all be of the multiple choice variety, 
with a response mechanism suitable for computer grading. One of the 
quarterly diagnostics will replace the mid-term examination and the last 
of them will be administered about one week before the standard final 
examination . 

8, At the end of each quarter the instructor will submit a 
diagnosis and recommendations based upon study guide responses, 
performance on Progress Checks, and quarter diagnostics. Possible 
recommendations include continuation of sequence, repetition of specific 
segments, further use of other program texts, additional tutorials, and 
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dropping out. 



Flow Chart 
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Results 



INTERNAL MANAGEMENT 



Internal managements the guiding of each student through each 
unit of course material, is provided by several components of the instruc- 
tional system. Study Guides, Progress Checks, and administrating instruc- 
tors are all part of this process of directing the student through the 
instructional materials, 

A management system should be dynamic and responsive. The system 
should evaluate a student’s response immediately, and modify the learning 
input accordingly, as depicted in the feedback loop below. The diagram 



Learning material 
Components 




shows exposure to learning material as input to the student, who then 
gives a response as output. The response is evaluated and the student 
is directed to the most appropriate learning component which modifies 
the input and completes the loop. The physics program has two levels 
of internal management which are described by this feedback loop. These 
provide management through both formative and summative evaluation. 
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The Study Guide is the primary agent of individual management in 
the present program; it is the instrument of formative evaluation and 
v decision making. The Study Guide reveals the correct answers; students 
compare their responses with those presented. On the basis of his 
evaluation, the student decides on various options offered by the Study 
Guide. These options incjude Information Panels, readings, videotapes, 
talking books, illustrated texts, core problems, enabling problem 
sequences, and core urime problems. This management level is illus- 
trated in the diagram below which is a specific case of the basic 
feedback loop. 
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Management of this kind is aimed at individualization of learning 
for each student according to his individual requirements. In effect , 
it provides a control and guide for independent study through branching 
programmed instruction, texts, and a variety of audiovisual support 
materials. The student’s activities are directed in accordance with his 
pet formance, at an item by item level. 

One of the goals set forth at the inception of the program was to 
establish optimal individualization with respect to media. The evalua- 
tion team sought to determine what media were most efficacious for a 
student with a known set of background variables. The disappointing 
result was that in the operating course they were unable to detect signi- 
ficant differences in performance due to media usage (see the section on 
Audiovisual Components), However, it is clear that media preferences 
ex is l i and this has become the basis of the present system of voluntary 
selection of audiovisual material. 

Progress Checks provide the best objective evidence of a student’s 
achievement. These are the summative evaluation instruments for internal 
management, providing remedial instruction or sources of corrective infor- 
mation after the need is identified. Administrating instructors use the 
Progress Checks as a basis for directing the student to various options. 
These options include remedial sheets with references to specific pro- 
grammed instruction, audiovisual components, individual tutorial sessions, 
and retests on equivalent Progress Checks, The diagram of this management 
level is shown on the next page. 

More general management considerations are discussed in the 
Management System Report, TR 5* 4a, 
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POLICING DEVICES 



The open format we recommend for Problem and Solution Books does 
not include the policing devices requested by the Academy staff. 

Security measures were incorporated into the system to insure 
that all students worked through all of the instructional material. 

We believe that such strictured design is an impediment to the 
self-paced and self-managed concept. True-false "policing questions, 
scrambled text format, and hidden answer monitoring devices ire all 
wasteful of students* time. We believe these also to be motivation 
dampeners , 

Moreover , a great deal of effort and expenditure of time and 
money was devoted to the conception and construction of such security 
measures— all, we feel, are without justifiable purpose. 
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CONSTRAINTS 



The development and implementation of any automated learning 
system will have to satisfy various constraints imposed by the nature 
of the institutions the nature of the subject matter, physical facilities, 
funds, available time, contractual arrangements, and the character and 
qualifications of the personnel associated with the project. 

It is a virtual certainty that any other self-paced program will 
encounter different constraints from those met in the physics program, 
and will require different accommodations from the course developers. 

Most of the constraints on the physics project have been described 
in various sections of this report- It may be useful, however, to 
list these constraints so that other course developers can see where 
they have greater latitude and where they are more strictured. 

Since the self-paced course had to cover the same objectives as 
the conventional S211 course at the Academy, the subject matter was 
rather rigidly prescribed. 

It was not desirable to allow Academy students to be penalized 
academically due to any deficiencies in the developing project. The 
Admiral ordered that midshipmen could not be deprived of any existing 
materials during the experimental phase. Thus, the textbook and addi- 
tional individual instruction (tutoring) were available to the experi** 
mental group. This lack of controlled conditions precluded precise 
experimentation . 

Academy policy required that grades be established on a norm- 
referenced basis. Since the program was developed on a criterion- 
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referenced basis, this policy reduced the relevance of a major course 
component— the Progress Checks, 

Accurate reporting procedures on the part of the students was not 
enforced. It is worth noting that Academy professors were much more con- 
cerned with their students 1 progress than with recording their progress — - 
an attitude which was fostered by a separation of contractor and "customer 

Class size at the Naval Academy is limited to twenty students per 
section. It was therefore not possible to show that more students can 
be served by one instructor through the use of the program. 

The final exams were designed by the ^c^demy faculty including 
instructors of conventional sections. Two consequences of this arrange- 
ment are that course objectives are only sampled, and the test favors 
the sections which cover the least material (see Course Evaluation for 
particulars) . 
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REVISION 



The revision procedure is central to program development* By a 
continual process of successive revisions, the course evolves into a 
progressively more effective instrument. E^ch operation of the course 
produces data for the next iteration in an empirical approach to optimi- 
zation . 

Major revision steps in the physics project have already been 
described for each course component. Generally, these entailed rather 
drastic reconstructions of materials and working philosophies to produce 
the current version* Such revisions do not have a systematic character — 
they are often inspired answers to unexpected problems* Now that a pro- 
totype has been developed, further revisions should entail relatively 
small perturbations in content and materials. 

Physics course designers treated these less extreme revisions with 
a basic methodology: every objective for which performance is unacceptable 

undergoes a refinement procedure whereby smaller steps toward the objective 
are provided together with additional learning materials and alternative 
paths . 

The first step of the procedure is to compile all available performance 
measures according to the associated terminal objectives. These include: 
test items from progress checks, the final exam, items from the Problem 
and Solution Books, the associated learning categories (zero-, one-, and 
multiple-step) , the averages on each item, the averages of the upper and 
lower halves of the student population on each item, and indicators to show 
when an audiovisual or information panel is directly associated with that 
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objective. A page from the Data Compilation index and the key to column 
headings for the 1969 tryout is included as an illustration* 

Validation of all items is the second step in the revision process* 
Each item is examined in the light of the performance data with special 
attention to those with the poorest performance. No arbitrary performance 
standards should be predetermined during this procedure because even very 
poor problem statements may be correctly solved by a majority of students. 
The converse situation of a good problem with a poor performance is even 
more prevalent* The reader is referred to the Test Items section for an 
extensive discussion regarding valid problem construction. Appropriate 
adjustments in the items may be made at this stage, including minor changes 
in sequence. A Sample Validation Page from the Revision Process Report is 
incorporated in this section. Notations should be made on the Data Corn- 
pi lation Sheets of items altered in the validation process. 

The third step in revision is to compile several lists which serve 
to identify those terminal objectives with poor performance. The percen- 
tages used to define these lists are arbitrary — they should be raised 
progressively as the course evolves: In list A, include objectives on 

which less than 70% of the students were correct on progress check and 
final exam. In list B , include objectives on which less than 70% were 
correct on progress cheeks and more than 70% were correct on the final. 

List C includes objectives which seem to require more review, having less 
than 70% of students correct on the final and more than 70% correct on 
progress checks. 

Finally , the listed objectives and the associated data are examined 
and various "actions" are taken which may include an increase in any or 
all of the following: core level problems and solutions, enabling problems 
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KEY TO SAMPLE 



PRETEST 



POSTTEST 



FINAL 



STUDY GUIDE 



(AV) 



O 
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OBJECTIVES INDEX AND DATA COMPILATION 



Explanation of Column Headings 



' 


Item 


Problem identified by volume 
letter and number 




CCU/L) 


Average percentage correct 

(In this column U/L does not apply) 


— 


Item 


Problem identified by volume 
letter and number 




L « C * 


Learning Category 




C(U/L) 


Average percentage correct 
(upper half /lower half) 


— 


Item 


Problem number 




M/F 


Multiple choice/ Fill-in 




C(U/L) 


Average percentage correct 
(upper half /lower half) 


— 


Item 


Problem identified by volume 
letter, grouping, and number 




One Punch 


Number of students selecting one 
answer to the item 




Two, Three, 


Number of students selecting two, 




and Four Punch 


three, or four answers to the ? tem 




E.O. 


Enabling objective for item 



The notation ‘ ! (AV) U in the Terminal 
Objective column indicates that 
audio-visuals are associated with 
that objective 
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SAMPLE 

DATA COMPILATION BY TERMINAL OBJECTIVE 
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Bl-22 

Bl-23 

Dl-8 



Segment 4 #14 



SAMPLE VALIDATION PAGE 



R1 R2 R3 R4 
54 47 21 6 



The problem is valid and well described. 

Two trivial changes to "clean-up" the solution may be made. That 
is, replace 0 with its equivalent of 30® * and eliminate the words 
", . .makes life easier and, . . " 

TO 13 



Segment 4 #19 



Rl R2 R3 R4 
37 51 35 3 



This is a satisfactory item as it stands. It would, however, 
be in a better sequence if it was preceded by the supplementary note 
on tension and by problem #11, and then followed by problem #18, 

T® 13 



Segment 4 #21 Rl R2 R3 Ri 

53 58 15 0 



This item is valid, however the statement of the problem is in- 
complete; i,e., it relies heavily on the diagram, and in doing so it 
may become ambiguous. 

Alternate answer C may be considered invalid. 

It is reconnnended that the statement of the problem be rephrased 
and alternate answer C be replaced. 



TO 16 



Segment 4 #22 



Rl R2 R3 R4 
62 50 11 2 



Alternate choice "C" of this item is confusing and ambiguous. The 
item becomes wholly satisfactory when choice "C" is replaced by "none 
of the above". 
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and solutions, information panels, test assignments, time available for 
study (provided by eliminating less important material) , audiovisual 
materials, and alternate study guide branches. Some of these actions 
are depicted on an illustrative Sample Revision Action Summary, Once 
the revisions are made, many items will be incorporated, removed, or 
rearranged. Thus, some record must be kept to relate the old items to 
the new as is shown in the Sample Item Location Sheet. 

For information panels and audiovisual materials, performance data 
are not sufficient criteria for revision (see Course Evaluation report) , 
Presumably, the information content of these materials does not change 
during revision, but changes in format, wording, and presentation should 
be made. Factors such as interest, appeal, and ease of reading or viewing 
must be considered. These points are elaborated in the sections on 
Information Panels and Audiovisual Materials, and in the Revision Process 
Report, TR 5.5, 
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SAMPLE REVISION ACTION SUMMARY 



POST TEST C KEY: IP = Information panel (number after IP = question 

number which follows) 

CL - Core level problem 

EL = Enabling level problem 

PR = Problem revision 

TE - Time expansion 

TEX - Critical time expansion 



T,0. 


LIST 


SEGMENT 


ACTION 


21 


A 


7 


IP 1, 5, 8, 11 / CL 9, 14 / TE / PR 9, 10 


22 


c 


7 


CL 14 / TEX 


23 


B 


7 


IP 15, 18 / CL 17 / TE / PR 14, 16, 18, 19 


25 


c 


7 


IP 24 / CL 23, 29 / TE / PR 4*(8), 18*(8) 








* Sag 8 


26 


B 


8 


IP 9 / CL 12 / TE / PR 5, 6, 7, 8, 9 


27 


c 


8 


IP 5 / CL 8 / TE / PR 11, 12, 15, 16 


29 


B 


9 


IP 6 / CL 10 / EL 8 / TE / PR 11, 12, 13, 14, 15 
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